Can AI in Health Care Be Trusted? That’s Up to Us

Getting your Trinity Audio player ready…

We have witnessed a dramatic change in the perception and acceptance of artificial intelligence in health care.

Three years ago, when ChatGPT launched and I was a cabinet secretary advising California Gov. Gavin Newsom on new technologies, conversations about regulating AI centered not only on its possibilities, but also its risks and guardrails. Patients wanted to protect their personal information, universities feared students would use it to cheat, and workers worried about being replaced by robots, chatbots, and algorithms.

While those anxieties persist today, we now have widespread adoption of AI by providers and consumers.

At California Northstate University, on whose board of trustees I serve, educators who initially had concerns about the algorithms that underpinned AI are now incorporating it into the curriculum. They understand that the mission of their medical, dental, pharmacy, and psychology schools is to prepare and train future providers to function effectively in an AI-driven world.

I have felt that shift in my personal interactions with health care, too. While my mother battled cancer in 2022, my knowledge of her disease and available therapies came almost entirely from her doctor. Four years later, while caring for my father during his own cancer battle, information about novel therapies and groundbreaking clinical trials was at my fingertips.

AI offered me knowledge and empowerment to make my way through our complex health care system during a difficult and emotional time. These elements are especially important to immigrants like myself for whom access to high-quality equitable care can be inhibited by linguistic barriers.

While I find AI’s potential to improve health care access and equity encouraging, I remain wary.  It was tempting, for example, to accept AI diagnoses and treatment suggestions for my father’s disease before corroborating them with a physician. When I consulted his care team, I learned the accuracy of the AI information was mixed.

Disclosure Protects Patient Trust

I am not alone in relying on AI while questioning its reliability.

Recent polling shows that one in four Americans have consulted AI for health decisions, while fewer than 1 in 10 actually trust it. And while more than three-quarters of adults are concerned about data privacy, nearly half have uploaded medical information to an AI tool.

Our society tends to embrace shiny new technologies before doing due diligence. Decades after the advent of social media, its corrosive effects on mental health and media literacy are only now beginning to be litigated. And when the internet first became widely available, web users had no reference point for sifting through the unprecedented volume of information they could access. Without firm guardrails and intentional engagement, we risk playing a similar game of societal catch-up with AI. This time, the stakes are even higher.

To unlock AI’s power to improve access to quality health care without undercutting trust in the system, the key is transparency.

History tells us that when providers act without patient consent, trust falters. Take the example of Henrietta Lacks, a Black woman with cervical cancer whose stem cells became instrumental to groundbreaking medical research after they were harvested and distributed worldwide without her consent. For Black Americans, that ethical breach represents one of many exploitative actions that have triggered profound distrust in the medical establishment.

Providers and policymakers can avoid seeming to be deceptive about AI by clearly explaining how the technology is being used. It could be a clinician’s note-taking app, a diagnostic tool, or data-sharing software like the California Department of Public Health’s PHNIX program, which crunches cross-sector data to give insights into public health strategies. Full disclosure will always be needed.

Moreover, inviting questions empowers California patients to be active participants in AI-assisted care, whether that involves a scheduling algorithm, a preventive screening tool, or a program that translates Medi-Cal materials for Californians with limited English proficiency.

Maintaining a Healthy Skepticism

Keeping an accountable human in the loop, both as a guardrail for accuracy and as a mechanism of trust building, is vital. Patients from diverse backgrounds care deeply about their relationships with providers, and they don’t want AI to replace their doctor.

Providers can retain that confidence and connection by assuring patients that AI is not a substitute for human scrutiny and expertise and carefully analyzing AI-generated content. This is especially important for future clinicians being trained in an AI world. For my colleagues at Northstate, the priority is to ensure their students are skilled at using AI without becoming overly reliant on it.

The next generation of doctors needs to approach AI responses with skepticism and pose prompts in several different ways. They should also test AI in low-stakes scenarios, asking a question they already know the answer to before trusting it with high-stakes inquiries.

With hefty investments in ethical AI emerging, a trust-first approach feels within reach, especially if developers reinvest a portion of their earnings in safety and training infrastructure. But regulation and investment, while important, are inadequate substitutes for our own critical engagement.

The Inevitability of AI Transformation

The health care industry cannot sit back and wait for legislators and AI developers to set responsible parameters. Providers, policymakers, and patients should test use cases, familiarize themselves with AI applications, and be vocal about what does and doesn’t work.

By using AI with intention instead of just theorizing about it, the health care system and the public writ large can help shape what is an acceptable use of AI in health care. We still have a degree of control over how AI is applied in this field, and it is vital that we act on it.

After all, the question is not whether AI will transform health care, but whether we will build the accountability structures necessary to make that transformation a good one.

In my father’s final days, I didn’t ask his care team whether they were using AI to evaluate his condition or explore ways to extend not just his life, but the quality of it. Now I wonder if I gave him the best possible care. I would have been comforted to know that his providers were using AI to supercharge their expertise. For families facing their hardest moments, knowing that every tool is being used to help a loved one is itself a form of healing.