For a long time, there was a serious problem with IT – it didn’t “tell a patient’s story,” according to Michael Pfeffer, MD. “It was just a bunch of pre-generated text.”

Now, that’s no longer the case. Thanks to the tremendous effort that’s been put into place in recent years, “We’re able to capture the patient’s words in a more accurate way than we’ve ever done, and that’s incredibly exciting,” he said during a recent 229 Podcast interview.

At Stanford Health Care, where he serves as Chief Information and Digital Officer, clinicians are leveraging large language models (LLMs) to ensure they’re making “the best decisions for patients in real time, with the best evidence,” which is, put simply, “changing the game.”

In the interview, Pfeffer shared thoughts on how creating a sandbox environment has benefited both users and leaders, why he’s so excited about ChatEHR, and what it means to be “uniquely Stanford.”

Michael Pfeffer, MD

One of the most common misconceptions about AI is that it’s new to medicine; as both a CIO and a practicing physician, Pfeffer knows that isn’t the case. “We’ve always had some form of AI; we were predicting things and using rule-based intelligence,” he said. The difference? “Now we have a super-advanced version of AI with capabilities that let us challenge the way we do things in informatics.”

LLMs, which are used to summarize, categorize, and generate text and photos, can sort through “huge amounts” of both structured and unstructured data, “in ways we haven’t been able to before.” And, as demonstrated by the saturated vendor market, AI tools are more accessible than in previous years, enabling individuals to experiment by writing prompts and asking questions.

Playing in the Sand(box)

That curiosity is absolutely critical, according to Pfeffer. “When you’re having discussions around how to solve problems with AI, that’s transformative. That’s what makes it fun.”

And in fact, that thinking is a hallmark of what it means to be “uniquely Stanford” and create a culture of innovation that permeates through the organization. “It allows us to think big and move toward outcomes that can be taken and disseminated.”

That was the strategy Stanford employed soon after the launch of LLMs, creating a space in which staff could experiment with different models and learn how they work. For leaders, it offered a valuable glance into how models were being used, and a chance to customize and automate based on those findings.

In one case, SecureGPT (Stanford’s term for the sandbox) was being used to record mental health interactions and provide a transcription and summarization, which would have taken “hours and hours” to do without the help of a voice file. “It’s saving us so much time,” he said.

That capability to marry the clinical record with LLMs in an efficient way paved the way for ChatEHR, which went live in the fall of 2025. Its impact was felt almost immediately, as it helped physicians get to know patients through prompts, rather than trying to read through hundreds of notes.

“If you’re admitting a patient for the first time, it's really helpful to dive into the record and make sure you pick up all the details that you need,” said Pfeffer, who believes “ChatEHR can summarize the hospital course better than I’ve ever seen anybody do it. That’s exciting.”

Learning and Iterating

Of course there are myriad security and privacy measures in place to safeguard data and ensure the correct record is being accessed. That, he noted, “significantly reduces the hallucinations. But we continue to monitor it. We continue to learn from what people are doing and how they think, and use that to develop AI-based automations and products that match those needs.”

How exactly does Stanford do that? One way is through MedHELM, a framework designed to decipher how LLMs are performing by soliciting user feedback. Importantly, it pulls from all records involving the patient, including HIEs, to provide a complete picture, and allows physicians to question trends. “It’s been amazing to see it in action and have people find things they maybe wouldn’t have found,” he said.

What’s just as exciting as having an EHR user interface is the ability to run sophisticated analyses, Pfeffer added. “Instead of just rules-based data, you can take a huge set of criteria and run it against the EHR data in real time, and produce output that you can then put back into the workflow.”

A practical use case? Leveraging ChatEHR to identify which patients are eligible to be transferred to another campus, a process that involves sifting through a huge amount of criteria. At Stanford, “we can take that criteria and run it against every patient in the ED, then flag the ones who are eligible, rather than having people do manual chart review,” he said. “That is not something we could do before.”

The Green Button

It is, however, something they plan to do in the future, as agentic AI is increasingly being used to alleviate the administrative burden, while also enabling physicians to spend more face time with patients. “That’s where we see a lot of value,” he said.

And what has made it possible is the groundwork Stanford has put into place, including a ‘Green Button’ that provides guidance for physicians in the absence of evidence-based guidelines by pulling data from the EHR. The initiative, which was the first-of-its-kind, has been a huge satisfier for users and is currently being piloted in primary care facilities.

“It’s growing to become more automated and integrated,” said Pfeffer, who believes it holds tremendous potential. “I’m a huge advocate for technologies that provide evidence to make sure we’re doing the right thing every time for the patient. When you boil this down, that’s the only thing that matters.”