1
INTRODUCTION

As artificial intelligence (AI) methods and capabilities have advanced over the past decade, interest in the application of AI to make meaningful contributions to disease prevention, management, and cure has surged. These advances in AI come at an opportune moment for health care as the field struggles with significant challenges, both long-standing and exacerbated by the COVID-19 pandemic. Concerns include but are not confined to rising costs, strained staff and staffing, limited and inequitable access, disparities in outcomes, growing disease burden, and persistent patient safety challenges. According to the most recent data available from the Centers for Medicare & Medicaid Services, national spending on health care reached 17.3% of gross domestic product in 2022 (CMS, 2024). A recent literature review reflected that 35% of physicians are experiencing burnout (Hoff et al., 2023). Nearly half of adults surveyed by the Kaiser Family Foundation through ongoing polling, updated in March 2024, reported difficulty in affording health care (Lopes et al., 2024). Disparities in care and outcomes persist as documented by a recent analysis of maternal outcomes that demonstrated that Black mothers in the top 20% of income distribution experienced a mortality rate of 4.3 deaths per 10,000, while White women in the lowest income quintile had a maternal mortality rate of 2.7 deaths per 10,000, a difference of nearly 60% that cannot be explained by access and resources (Kennedy-Moulton et al., 2023). Estimates from the Centers for Disease Control and Prevention (CDC) indicate that the prevalence of diabetes in the U.S. population continues its upward trend, growing from 10.3% in the period between 2001 and 2004 to 13.2% in the period between 2017 and 2020 (CDC, 2024). And, since the publication of the To Err Is Human (IOM, 2000) describing the staggering number of deaths each year due to medical errors, progress in improving patient safety has been inadequate (Bates and Singh, 2018). Clearly opportunities for improvement abound.

In the last several years, AI tools have shown the potential to ameliorate many of these systemic challenges (e.g., improved accuracy and efficiency, cost

reduction, and staff augmentation) and may also play a role in truly solving them (Cutler, 2023). While the health care field is still at the beginning of integrating AI capabilities into clinical practice and administration, early tools and experimentation have yielded promising, and in some cases outstanding, results. The most mature efforts in integrating AI in clinical care may be in radiology where it has been employed across almost all subspecialties and modalities over the past decade (Yordanova, 2024). There is intense interest and investment in AI-powered products using ambient listening for clinical notetaking, which show promise but are not yet established (Tierney et al., 2024). Meanwhile, AI-powered tools have proven effective in performing many administrative tasks by automating documentation and other routine tasks (Abdelhady and Davis, 2023). AI is also streamlining complex health care supply chains by getting the right resources to the right people faster. For example, in Ghana and Rwanda, AI-enabled drones are delivering life-saving vaccines, medications, and blood products to remote and underserved regions (Krittanawong and Kaplin, 2021). By leveraging AI tools trained across interconnected, highly secure, standardized and de-identified datasets, researchers, clinicians, and innovators are exploring how AI can both improve the current care delivery system and create opportunities for improved patient outcomes (Halamka and Cerrato, 2021). Indeed, as AI continues to improve and mature, it holds promise to impact many long-standing challenges and to create new opportunities to markedly improve health care delivery and health outcomes.

Administration, logistics, patient navigation, preventive care, clinical care, and virtual care modalities are all poised for marked improvements due to cutting-edge AI applications. For example, AI-powered tools show great promise to increase the speed and accuracy of pathology results interpretation (Greely et al., 2024; McGenity et al., 2024). In direct clinical care, AI-assisted procedures—such as AI-assisted colonoscopies (Wallace et al., 2022) that promote fuller examination of the colon—have significant potential to support higher quality of care delivery and improvement in health outcomes for patients across the world, resulting in less disease and fewer complications for more people. AI models are being used to predict risk of clinical deterioration in patients in intensive care units, aiding in more rapid interventions and resulting in decreased mortality (Escobar et al., 2020). AI-enabled devices and systems, running continuously as seamless end-to-end solutions, may allow both patients and their care teams to detect and prevent diseases earlier than conventional testing—such as through AI-enabled ECG analysis that, when accompanied by leadership support, shows promise to identify early heart failure before symptoms develop (Yao et al., 2021). AI has also shown the potential to transform personalized medicine, making care faster,

more precise, and more efficacious, such as algorithms that analyze genomic data to precisely anticipate a patient’s response to specific therapeutic interventions—allowing patients to receive more effective and potentially safer treatments (Tao et al., 2020). Moreover, generative AI shows early promise in simplifying pathology reports for patients navigating the health care system by explaining test results, outcomes, or billing (Steimetz et al., 2024) and acting as a kind of interactive health care “encyclopedia” (Reddy, 2024) and may be capable of providing medical language interpretation in the future.

The overwhelming majority of individual health is managed through self-care and caregiving in the community, outside of the traditional health care delivery system. Patients and caregivers are increasingly utilizing personal and health care certified AI tools, applications, and devices to support self-management. Correspondingly, health care research and operations personnel are developing new tools to collect and analyze these important health data. AI-enabled wearables and remote monitoring tools that assess sleep, breathing, cardiac rhythms, and ambient voice data (Bajwa et al., 2021) have the potential to support self-care and to connect patient experiences more closely than ever to medical research, helping to close the gaps that exist in delivering evidence-based medicine.

There is great interest in using available AI tools to improve business operations and clinical care; furthermore, AI experts anticipate that ongoing scientific advances will lead to AI-enabled capabilities that are currently not possible (Anderson and Rainie, 2023). AI-assisted health research is exploring new capabilities in the biomedical sciences that accelerate drug discovery and design, identify novel targets, and help assess toxicity (Mullowney et al., 2023). Clearly, the potential for AI to improve health and health care is substantial, as is investment—both public and private—in AI research to drive the development of new tools and techniques. In 2021, U.S. non-defense agencies allocated $1.5 billion to AI research, while the private sector spent more than $340 billion (Ahmed et al., 2023b). The drive to continue to innovate is creating competition across the industry and the world (O’Brien, 2024; YOLE Group, 2024), and in the current unregulated environment, the incentives for personal gain could stymie the public good.

Concomitant to the drive and investment to rapidly advance AI techniques, tools, and adoption, there are important challenges and limitations that must be addressed to fully realize the promise and potential of AI in the health sector. Some examples include risks to safety, security, equity, and accountability. Additional existential threats such as misinformation, job loss, and widespread surveillance may be introduced by AI systems (Anderson and Rainie, 2023). From the perspective of patient safety and effectiveness of care, failure of an AI tool to accurately predict target outcomes could result in care that is of lower quality

than care delivered without AI support. For example, a model that inaccurately predicts a negative patient condition (e.g., infection) in an acute care setting could lead to delays in care. Such a situation was reported when a validation study of an inpatient sepsis prediction model embedded in a widely used electronic health record demonstrated that, in practice, the model resulted in poor discrimination, missing some cases and over-alerting in others (Wong et al., 2021). The extremely large datasets used by AI also present privacy and security risks to patients. In 2024, multiple extremely large health data breaches were reported to the Office for Civil Rights, exposing data on hundreds of millions of individuals to nefarious actors (OCR, n.d.).

Health disparities and equity concerns can be created or exacerbated by AI. For example, the choice of data to include in a model can inadvertently build bias into the model. In the review of a model used to guide provision of resources to patients in need of supplemental health supports, Black patients were identified for additional care at a rate half that of White patients who were not as sick; this was due to the inclusion of health care costs as a proxy for illness burden in the model (Obermeyer et al., 2019). Another way in which inequities may be introduced is in access to the benefits of AI. Highly resourced organizations, such as academic medical centers, may be more able to implement new AI tools (Wu et al., 2023) than smaller, more poorly resourced organizations, such as federally qualified health centers, furthering the digital divide.

With the rapidly expanding use of AI in the health sector, accountability for development, testing, maintenance, and use also presents challenges that require careful attention (Davis et al., 2024). Newer AI methods may make transparency and explainability more difficult (Saeed and Omlin, 2023), reducing trust in the systems. Yet there are currently no national standards for assessing health care AI models, their outputs, or their governance, from their development and training to the necessary vigilant monitoring post implementation, to the outcomes they produce.

To realize the benefits, the risks associated with health AI must be addressed to ensure both high-quality, evidence-based care and to promote ongoing trust in the health system. The National Academy of Medicine’s (NAM’s) AI Code of Conduct (AICC) framework, presented below, was developed with guidance from the AICC steering committee and is designed to align the health field and to catalyze collective action to ensure that the transformative potential of AI to advance health, health care, and biomedical science is realized, while upholding the “highest standards of ethics, equity, privacy, security, and accountability” (NAM, n.d.). The AICC steering committee developed for public comment a draft AICC framework (Adams et al., 2024), comprising a set of Code Principles, informed

by a literature review, harmonized with national and international guidance, and mapped to the NAM’s Learning Health System (LHS) Shared Commitments (McGinnis et al., 2024). In the spirit of the NAM’s Shared Commitments, the Code Commitments represent a set of simple rules or expectations intended to aid in the development and application of AI in health and health care. This publication represents a continuation of that work and delves deeper into the opportunities and challenges presented by health AI, presents updated Code Principles and Code Commitments, considers the requisite new actions and collaborations for impacted parties, and poses a set of priority actions for consideration by the various actors in the health arena to realize the vision of an LHS enabled by AI and guided by the Code of Conduct framework.

This page intentionally left blank.