Artificial Intelligence (AI) is increasingly being developed and used in all aspects of health care in ways that will fundamentally affect how care is delivered, how new treatments are developed and deployed, and how patients and clinicians interact. This rapid technological advancement can solve some of the greatest challenges in health care but also raises significant risks surrounding reliability, bias, and privacy. Policy has been developing much slower than technology, leaving uncertainty regarding the government’s oversight of AI in health care.

The Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (EO), signed by President Biden on October 30, 2023, provides some clearer direction regarding US policy. It establishes a framework for managing AI risks, directs federal action to regulate the use of health AI systems, and guides the development of tools to advance AI innovation across sectors, including health.

The EO comes as medicine is emerging as an information science, and health technology and AI are increasingly being used to improve efficiency in care delivery, increase access to care, improve diagnosis and treatment, and engage patients. However, the risks of AI-enabled technology in health care include exacerbating health disparities, compromising the privacy of intimate personal information, and triggering improper clinical decisions that could be a matter of life and death. While the EO closes some of the policy gaps by providing a framework for AI, it falls short in providing the details or binding requirements that may be necessary to build trust in the use of AI to meet the needs of health care delivery, discovery, and administration.

The EO outlines eight guiding principles and priorities to advance and govern the use of AI, all of which apply to health care: (1) ensure safe and secure AI technology; (2) promote responsible innovation, competition, and collaboration; (3) support American workers; (4) advance equity and civil rights; (5) protect American consumers, patients, passengers, and students; (6) protect privacy and civil liberties; (7) manage the federal government’s use of AI; and (8) strengthen US leadership abroad, promoting safeguards so that AI technology is developed and deployed responsibly. In addition, the Department of Health and Human Services (HHS) is directed to take several specific actions.

The EO’s Impact On Health Care

HHS has already engaged in specific policy activities to oversee AI-enabled health technology. For example, the Food and Drug Administration has issued guidance on AI-based software as a medical device. Also, the Office of the National Coordinator for Health Information Technology has proposed regulation on transparency related to predictive decision-support interventions incorporated into certified electronic health records. However, the federal government has not taken a comprehensive approach to developing, testing, and using AI in health care. This EO calls on HHS to establish an AI Task Force within 90 days and develop a strategic plan for responsible AI deployment within one year. The strategic plan is focused on the following areas:

• Development, maintenance, and use of predictive and generative AI-enabled technologies in health care delivery
• Safety and real-world performance monitoring of AI-enabled technologies
• Incorporation of equity principles, including protection against bias
• Incorporation of safety, privacy, and security standards into the software development lifecycle
• Development and availability of documentation to help users determine appropriate and safe uses of AI in local settings
• Coordination with state, local, tribal, and territorial health and human services agencies to advance positive use cases and best practices for the use of AI
• Identification of uses of AI to promote workplace efficiency and satisfaction

The strategic planning will likely take a year, leaving health care organizations and health technology companies that are developing, considering, or using AI-enabled technologies uncertain for the near term.

However, there are some specific areas that HHS is directed to address sooner. These include most notably the establishment of an AI Safety Program by October 2024, which, in partnership with voluntary federally listed patient safety organizations and building on their previous work, would establish a common framework for approaches to identifying and capturing clinical errors resulting from AI deployed in health care settings. The program would develop specifications for a central tracking repository for associated incidents that cause harm to patients, caregivers, or other parties; analyze data and generate evidence to develop informal guidelines to avoid these harms; and disseminate them to appropriate stakeholders.

HHS is also called on to develop specific strategies on quality, non-discrimination, and drug development. Specifically, HHS is directed to:

• Develop a quality strategy, including premarket assessment and post-market oversight of AI-enabled health care technology performance against real-world data
• Advance compliance with federal nondiscrimination laws by HHS providers that receive federal financial assistance and increase understanding of how those laws relate to AI
• Develop a strategy for regulating the use of AI in drug development

The EO also provides funding opportunities, including requiring the National Institutes of Health to prioritize grantmaking and cooperative agreement awards to promote innovation and competition. Furthermore, broader federal actions called for in the EO related to privacy, including developing privacy-enhancing technology, cybersecurity, and non-discrimination, will likely impact health care.

What This EO Does Not Do

In general, the EO allows the Biden Administration to coordinate efforts across various agencies, set an overarching policy, and clearly state priorities regarding complex issues that cut across multiple sectors. Specifically, an EO directs action by federal agencies, but it can only require these agencies to act consistent with their existing authority and budget. Moreover, regulation or even legislation would be required to change any requirements for the private sector. For example, the EO describes how federal agencies must protect data privacy but does not provide rules to guide the private sector on data protection. In the health care sector, any changes in health data privacy would require HHS to publish guidance or propose regulation changes or Congress to pass legislation that would protect the privacy of health data maintained outside of Health Insurance Portability and Accountability (HIPAA) protections, such as in consumer-directed health technology.

Another EO limitation is that it is not necessarily enduring. A new President with different priorities could revoke this EO — a limitation that is particularly pertinent because the implementation will take time and may not be completed before January 2025, potentially the end of President Biden’s time in office. On the other hand, many issues and policies addressed in the EO presumably have bipartisan support.

What’s Next?

We anticipate significant action by the federal agencies as they implement the EO. As it relates to health, the key milestones will be the safety reporting program and regulatory updates, some of which are in progress. The most critical step ahead will be the strategic plan HHS will develop within one year. This plan will set in motion the approach to regulating AI in health and health care and identify the priorities and actions that HHS will take to promote AI innovation and minimize the risks.

This article by Jodi G. Daniel and Harlan Krumholz was published on November 17 by Health Affairs Forefront and is reprinted with permission. Copyright © 2023 Health Affairs by Project HOPE – The People-to-People Health Foundation, Inc. All rights reserved.

Jodi G. Daniel

Jodi G. Daniel is a leading digital health lawyer and policymaker. She is a partner at Crowell & Moring, where she leads the firm’s Digital Health Practice and is also a managing director at Crowell Health Solutions and a director at C&M International. A former lead policymaker at the US Department of Health and Human Services, Daniel has deep understanding of the digital health market and cutting-edge health innovation, including artificial intelligence and machine learning, remote monitoring tools, electronic health records, mobile health applications, digital therapeutics, health data platforms, and data analytics tools. Read More

Harlan Krumholz

Harlan Krumholz is a cardiologist and scientist at Yale University and Yale New Haven Hospital. He is the Harold H. Hines, Jr. Professor of Medicine. He is a leading expert in the science to improve the quality and efficiency of care, eliminate disparities and promote equity, improve integrity and transparency in medical research, engage patients in their care, and avoid wasteful practices. Recent efforts are focused on harnessing the digital transformation in health care to accelerate knowledge generation and facilitate the delivery of care aligned with each patient’s needs and preferences. Read More