key: cord-0252448-w1252h98 authors: Williams, Carmel title: A Health Rights Impact Assessment Guide for Artificial Intelligence Projects date: 2020-12-03 journal: Health Hum Rights DOI: nan sha: 286c28d0eb02a79af5915bece71d3d36790b9d92 doc_id: 252448 cord_uid: w1252h98 nan In 2019, the World Health Organization (WHO) introduced a guideline on how to use digital technology to strengthen health systems. 3 The guideline provides useful indicators for assessing some of the impacts of AI on health systems, but it fails to locate the centrality of health systems to the fulfilment of the right to health. The guideline followed a resolution brought to the World Health Assembly in 2018 that recognized the value of digital technologies (including AI) and their capacity to advance universal health coverage and the SDGs. 4 However, the guideline concedes that enthusiasm for digital health has seen many short-lived implementations, an overwhelming diversity of digital tools, and a limited understanding of their impact on health systems and people's well-being. 5 It stresses the need to evaluate the positive and negative impacts of proposed digital health technologies and to ensure that such investments do not inappropriately divert resources from alternative, nondigital approaches and thereby increase health inequities. 6 The guideline advises that digital health technologies should complement and enhance health system functions, rather than replace the fundamental components needed by health systems, such as the health workforce, financing, leadership and governance, and access to essential medicines. 7 It calls for an assessment of the health system's ability to absorb digital interventions and warns that new technology must not jeopardize the provision of quality nondigital services in places where digital technologies cannot be deployed. It demonstrates the assessment of various applications of health-related technology based on effectiveness, acceptability, feasibility, resource use, and "gender, equity and human rights." The guideline encourages technology developers to work with users and to think broadly about context both within and beyond the health system, as well as to consider whether a given digital health intervention will improve universal health coverage. Although human rights are included with the "gender, equity and human rights" component for impact analysis, the specific indicator selected to assess this component is limited to the technology's impact on equity. 8 But equity-important as it may be-is only one human rights consideration. It is also necessary to examine other key principles of the right to health when assessing health interventions. 9 Although the WHO guideline examines various components of a health system when the component is directly affected by the technology, it fails to systematically examine the whole health system to identify any less obvious, indirect impacts of the proposed new technology. In response, this paper presents an expanded tool to help states and other actors undertake a right to health impact assessment prior to implementing AI projects. The tool, informed by the WHO guideline, is a refinement of an earlier impact assessment tool of aid-funded health projects in low-resource settings. 10 It accommodates additional considerations necessary when AI health projects are under development. It explores possible impacts, specifically on the right to health, moving beyond the civil and political rights most frequently associated with digital health, big data, and AI-namely, data privacy and protection, security, and algorithm transparency. It is a guide that provides a sample of the type of questions across the health system that need to be explored-but each project will need its own context-specific adjustments. The right to health and health systems Because the health system is the core institution through which the right to health can be realized, governments and other agencies have a duty to ensure that health systems are enabled to fulfill people's entitlements to available, accessible, acceptable, and quality health services (AAAQ). 11 Accordingly, governments have a human rights obligation to ensure that health systems are never weakened but rather continually improved as part of their progressive realization duties, as detailed in General Comment 14 of the Committee on Economic, Social and Cultural Rights. 12 One way to prevent a weakening of the health system while demonstrating a commitment to the progressive realization of the right to health is to carry out c. williams / perspective, big data, technology, artificial intelligence, and the right to health, 55-62 N U M B E R 2 Health and Human Rights Journal 57 human rights impact assessments prior to adopting and implementing policies and programs. 13 This applies to projects relating to digital health, projects driven by AI (irrespective of whether they are government or nonstate initiatives), and projects driven by local funding or through international assistance and cooperation. In order to conduct a health rights impact assessment on a health system, it is convenient to compartmentalize the system to enable impacts to be measured across its many functions. A useful schematic devised by WHO identifies the component parts that contribute to the delivery of health care: health services and facilities; health workers; health financing; medicines, products, and other supplies; health information systems; and management and governance. 14 Importantly, there is more to a health system than these technocratic elements: people and communities must also be included, as the right to health entitles them to participate in a meaningful way in the planning, delivery, and monitoring of health care and health promotion. Human rights-based approaches to health care and health projects promote the active engagement of people who will be using services, as well as the understanding that people are legally entitled to these services as a function of their right to health. Without people's participation, health services cannot achieve AAAQ for all. A health rights impact assessment is a systematic examination of a project, undertaken prior to its implementation, to anticipate the effect that it will have on human rights and health, including and extending beyond its own project-related goals. It should not be confused with, nor replaced by, a needs assessment, which is a narrower exercise that does not assess risks. A health rights impact assessment predicts immediate and longer-term impacts on the whole health system by examining each of the system's component parts and assessing the ways in which the project could strengthen or weaken that component. If risks are identified, an impact assessment considers ways to mitigate them. The purpose of such an assessment is at least twofold: it aims to strengthen the project by ensuring that it is in alignment with the health system and its governing strategies and plans; and it aims to strengthen the health system by helping design projects that will be sustainable and contribute to the protection and fulfilment of health rights. 15 Conducting a health rights impact assessment when a project is being designed can help governments adopt and implement policies, programs, and projects that will best meet their obligations to take deliberate and concrete steps toward the progressive realization of human rights. 16 It serves a further purpose as well, by promoting engagement with the key features of the right to health, outlined in Table 1 Table 1 . Key features of the right to health c. williams / perspective, big data, technology, artificial intelligence, and the right to health, 55-62 Health and Human Rights Journal AI for health care in low-resource settings Even before the extraordinary pressures of the COVID-19 pandemic, health systems worldwide were facing challenges, including greater demands for services with the rising burden of disease, increasing costs, and poor productivity and overstretched human resources. It would therefore be of great benefit to health systems and communities if technological advances could help reduce burdens on systems and health care costs while increasing accessibility and equity. 18 In an opening address to the "AI for Good Summit" in 2019, the secretary-general of the International Telecommunications Union, Houlin Zhao, urged the audience to "turn [the] data revolution into a development revolution." 19 To achieve this revolution, though, it is imperative that the development context is fully understood and reflected in the data solutions. Development has a long history of failed projects, especially those dependent on technology. 20 Enthusiastic donors can be persuasive partners when seeking to test new technologies in low-resource settings, and governments in these settings are presently indicating that they are "open for business" when it comes to AI partnerships. 21 Not only are the well-known pitfalls arising from a lack of ongoing technical or health worker support, or funding for maintenance, present with AI-based technology partnerships, but additional traps as yet unknown can arise from downstream data ownership, sharing, and reuse. 22 AI is being used in health care in various ways, including in diagnosis (especially imaging), patient management, treatment (for example, robotics in surgery), and new drug development. 23 In the wake of COVID-19, technology is also playing a large part in contact tracing, where it monitors the spread of the epidemic, and in the race to develop a vaccine. 24 But many of these uses demand a level of technical capacity well beyond that available to health systems in low-resource settings. Even if the technology is designed elsewhere and imported, its ongoing use requires an adequate, well-trained, and available workforce; infrastructure (including, at the very least, electricity and internet); and accessible health facilities so that the benefits of such advances are equitably available to all people. Designing data-driven technological projects for health care in low-resource settings requires a detailed understanding of their challenging contexts; otherwise, the interventions will almost certainly be inappropriate or unsustainable. It is difficult to acquire such an understanding from afar. But even locally developed AI-based technological solutions can fail to respect and protect human rights if they are not supporting the local health system in meeting the health rights of the people in its jurisdiction. 25 Thus, regardless of whether AI health projects are being introduced in a development context (a focus of AI for Good) or in high-income countries, it is imperative that systematic health rights impact assessments are undertaken and that they are broad enough to anticipate impacts on the health system components, as well as on civil and political rights relating to data privacy, ownership, and security. Adapting a health rights impact assessment tool for technology projects Presented in Table 2 , this assessment tool is framed to guide the development of AI for health projects that comply with a right to health framework in local contexts. Each of the proposed questions is linked to at least one of the key features of the right to health. Therefore, indicators are selected to assess impact on the health system and on the right to health. This perspective has presented a rationale for undertaking a right to health impact assessment before implementing AI health projects in high-or low-resource settings. Such assessments are in keeping with the United Nations' draft business and human rights instrument to regulate the activities of businesses and transnational corporations. 26 The tool in Table 2 demonstrates the range of questions that need to be addressed before implementing AI projects to determine how a new technology might affect the health system and, therefore, the right to health. c. williams / perspective, big data, technology, artificial intelligence, and the right to health, 55-62 To determine how the technology will be supported when IT partners exit AAAQ 7 Has the community been consulted to assess the technology's acceptability and accessibility? To ensure the participation of the population in designing the project, its implementation, and its monitoring Participation Acceptability and accessibility Transparency and accountability Health workforce 8 Have health care workers been consulted on whether the project aligns with the national health workforce strategy? To ensure that state plans and strategies, health care management, and health care workers been consulted about the technology Participation Availability (of health workers) 9 Could the project affect the number of health workers available to meet primary health care obligations or core obligations? To ensure that the new technology-driven project will not draw health workers away from other essential services Obligations of immediate effect AAAQ 10 What cadres of health workers will use the technology? To ensure that the use of digital technology is for tasks already defined as within the scope of practice for the health worker (WHO recommendation) Availability (of health workers) 11 How will health workers be trained and provided with ongoing support in their use of the technology? To consider who pays for training, ongoing support, and the cost of data: Is this sustainable? AAAQ Quality (of health care) Health information systems 12 Were departments and hospitals consulted on whether the project strengthens the present health information system? To ensure that departments and divisions in the health system and referral hospital are consulted prior to the technology's design Participation Quality 13 How does the project collect patient data, and can the data be integrated into patient records and the broader health information system? To ensure that the data are stored securely and integrated within the health system Health system strengthening Laws, norms, and standards Privacy health system-even though they are crucially important-but also assess other key right to health principles, including participation, accountability, equality and nondiscrimination, non-retrogression, and international cooperation. It is not enough for developers of a new AI application to claim that their application will address one health service and will therefore "help achieve SDG3 and universal health coverage"; without a right to health impact assessment, there can be no confidence that this is a likely outcome. Similarly, all human rights are interrelated and indivisible, which means that a rights-based app assessment must look beyond the health sector to determine how the technology could also affect other rights, including those related to privacy, confidentiality, and security. The long-term sustainability of a technology-based business depends not only on states' and businesses' fulfillment of their obligations to protect human rights but also on the development of products that service providers find useful, affordable, efficient, and acceptable to rights holders, including men, women, and children. These criteria apply whether the technology is state or nonstate owned and developed. Health systems and the right to the highest attainable standard of health Health systems and the right to health: An assessment of 194 countries World Health Organization, 2019), p. ix. 4. World Health Assembly p. ix. 6. Ibid., p. ix. 7. Ibid., p. xi. 8. Ibid The right to health supports global public health Using health rights to improve programme design: A Papua New Guinea case study Using a rights-based approach to avoid harming health systems: A case study from Papua New Guinea World Health Organization, Everybody's business: Strengthening health systems to improve health outcomes Impact assessments, poverty and human rights: A case study using the right to the highest attainable standard of health Artificial intelligence, machine learning and health systems Human Rights, Big Data and Technology Project blog The white man's burden: Why the West's efforts to aid the rest have done so much ill and so little good Growing inequality and risks to social rights in our new data economy Artificial intelligence and deep learning: Radiology's next frontier? Using artificial intelligence to reduce the risk of nonadherence in patients on anticoagulation therapy Digital phenotyping and digital psychotropic drugs: Mental health surveillance tools that threaten human rights AI's role in COVID-19 poses human rights risks beyond privacy 26. Office of the United Nations High Commissioner for Human Rights, OEIGWG chairmanship revised draft 16.7.2019: Legally binding instrument to regulate, in international human rights law, the activities of transnational corporations and other business enterprises This work was supported by the UK's Economic and Social Research Council (grant number ES/ M010236/1).