This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

Today on This Week Health.

the providers that were on the ground, prior to NLP, were spending about 80 percent of their time combing through the chart, trying to figure out, does this person really have an SDUH need, and then 20 percent making an impact.

With this tool, It was just right there in their face, and they can go directly to the note where it highlighted the, instance where that NLP insight was picking up something. And so it's flipped it from 80 20 to 20 80,

Welcome to Town Hall, a show hosted by leaders on the front lines with interviews of people making things happen in healthcare with technology. My name is Bill Russell, the creator of This Week Health, a set of channels and events dedicated to keeping health IT staff current and engaged.

We've been making podcasts that amplify great thinking to propel healthcare forward for over five years, and we want to thank our show sponsors. who help to make this happen. Armis First Health Advisors, Meditech, Transcarent, and UPerform. We thank them for investing in our mission to develop the next generation of health leaders.

Now, on to our show.

welcome to the This Week Health Town Hall Conversation. I'm Reid Stephan, VP Health System, and I'm joined today by Dr. Nirav Shah, who is the Medical Director of Quality Innovation and Clinical Practice Analytics at NorthShore Edward Elmer's Health. Thank you. And he's going to talk today about some really interesting solutions they've developed with some partners around using natural language processing to extract social determinants of health information from unstructured data in the EHR, and using this data to specifically identify issues like housing and food insecurity, transportation issues, domestic violence and stress, which in turn then helps social workers in the emergency department have more accurate and actionable data.

And so really excited to kind of unpack this a little bit more, Dr. Shah. But first, if you'll just take a minute and introduce yourself, share a bit about NorthShore, and then maybe a little more detail about this solution that you've helped develop.

Thank you, Reid, for having me. So I'm an infectious disease physician by training and in the midst of changing my role so that I'm on the innovation, IT um, and research teams.

North Shore Edward Elmer's Health is a product of essentially a rapid fire set of three or four kind of mergers in the last few years. We are located in northern Chicago and the surrounding suburbs. We have nine hospitals, 300 sites of care. We have about our catchment is about a third of Illinois, just to give you a sense of the size of our organization, we were also one of the first sites on Epic, one of the first sites to data warehouse all our data, and we're one of the first sites to be HIMSS 777, just to give you some insight kind of, to our.

technology and kind of where we're at.

Yeah, very impressive. Can you elaborate a little bit on some of the initial challenges that you faced when you were trying to extract this SDOH data from unstructured EHR data?

So we were, actually asked by our executive leadership to figure out how to extract SDOH elements.

And push it into a workflow within two years and it was tied to an internal grant. So we went on a wide search to figure out how to do this. We talked with colleagues, with mentors with companies, consulting companies vendors and we landed on a company called Linguamatics at the time.

It's been bought by IQVIA and they had a hybrid approach to extracting NLP. It was kind of a rules based plus an ML on the backend. And what was interesting was they actually had a set of queries already for SDOH. So at that time, this is pre gen AI and chat GPT and all of this it was either rules based or it was advanced kind of analytics, ML or deep learning.

And it was very difficult to kind of scale these types of things beyond a single or two use cases. So we went with this product and the challenge is I think essentially the pipeline challenge. figuring out how to extract the data, to process it, and then to push it back into EPIC in a way that's meaningful to our frontline team members.

And that that took us some time to figure out, but we figured out how to do that in a very meaningful way that I think we can discuss at length. Okay.

I'm curious, how did you kind of hone in on these specific domains within SDOH that you chose to focus on?

When we started this project there were a few places in our system where we were starting to extract H2H elements, through questionnaires and having our clinicians essentially ask.

And these were also areas that were tied to what social workers could potentially impact. And that's how we kind of decided that we were going to extract eight or nine kind of insights. Basically to target the SQH gaps that may exist. And so it was helpful that those same gaps were also available kind of as pre existing queries.

So we just had to kind of modify, enhance, validate retrospectively and prospectively. And that allowed us, a very quick way to kind of get moving with the items that we wanted to extract insight from.

Yeah. And as you've done this, what were some of the, maybe the most surprising insights or trends that you uncovered through this

tool?

So, a few things. I think in pushing this into a workflow understanding the importance of trust in this system and it not being a black box, I think was absolutely critical. So, just to give you some context, we Extracted the insights. It's a six month look back. I mean, essentially, you push it back into a column that the ED social workers already have, that they're leveraging to determine which patients to target.

And pre NLP, we were using various proxies that were not very good. It was what's your risk of readmission, risk of mortality. And did a nurse identify something? So you know, if the nurse identified there's some issue that has a high likelihood that, there's something that could be addressed.

The other things that were not ideal proxies. And essentially, we then pushed directly into the workflow that this was an S2H gap that needed to be addressed. So very quickly, we understood since we had access to the infrastructure of how these queries were essentially built and we could tweak it.

We found that one of our queries around depression some of the sub components of it were weight gain, weight loss. And we found that this was flagging every single pregnant patient. And it was quickly noticed by the ED social worker team. And we built a pipeline where we could continuously prospectively validate this so that any kind of issue that came about, our data scientists would know very quickly.

So as soon as they brought that up, we quickly made a change to the query. Cause in questioning all our pregnant patients while there may be at a risk for depression not all of them were actually depressed. So we quickly tweaked the query. And as a result, that kind of fixed that issue that existed within this.

And I think that was like one of the instances where that trust just really got solidified in this tool that it was something that was responsive to the evaluation by the on the ground team that are ED social workers. Okay,

and maybe just to dive into that a little bit deeper can you talk a little bit more about how the tool integrates with the workflows?

You, you discussed that earlier. And what's been the feedback from the clinical staff using this?

Yeah, in the patient column that the ED software workers use they basically, It's just right there. It just, it says identified by Linguamatics. Linguamatics was essentially the product that the company that got bought out by IQVIA.

And so they could click into it and it would show them all the different SUH elements that were available. And our Epic analyst did something very interesting where those insights, you know, where it listed there's a transportation issue or domestic violence type issue. You can click on that within that print group, essentially, and it would take you to the encounter or to the note where that insight was identified.

So, the providers that were on the ground, I like to kind of think about this as, they typically, prior to NLP, were spending about 80 percent of their time combing through the chart, trying to figure out, does this person really have an SDUH need, do they have a need that we can impact, and then 20 percent making an impact.

With this tool, It was just right there in their face, and they can go directly to the note where it highlighted the, instance where that NLP insight was picking up something. And so it's flipped it from 80 20 to 20 80, where they were, they spent 20 percent of their time. Essentially doing that chart review to understand what's going on with this patient, and then 80 percent of the time making that impact.

So, we suspect anecdotally that there's significant amount of efficiency gains that were related to this. We're actually in the midst of writing a paper specifically on this, so I, understand this. So, the, that's one of the key integrations is, I mean, there is the, there's the data pipeline where we essentially take.

Insight which is an on prem solution and essentially every night everything in EPIC goes into our EDW our enterprise data warehouse. And then we essentially run this algorithm on all the nodes every night and then we push it back into EPIC. And so that, that's one component of the integration.

The other component is really being able to find the actual node where this came from. So two distinct kind of integrations that I think were critical.

📍  We'll get back to our show in just a minute. Having a child with cancer is one of the most painful and difficult situations a family can face. In 2023, to celebrate five years of This Week Health, we have partnered with Alex's Lemonade Stand all year long with a goal of raising 50, 000 from our community.

We've already achieved that goal and we've exceeded that goal by 5, 000, so we're up over 55, 000 for the year. We want to blow through that number. We ask you to join us. Hit our website in the top right hand column. You're going to see a logo for the lemonade stand. Go ahead and click on that to give today.

We believe in the generosity of our community and we thank you in advance. Now back to our show. 📍   📍 Yeah, that's really impressive. What have been some of the patient care outcomes that you've seen improved by this? Do you have any examples you can

share?

Again, it's anecdotal. We're working on a paper actively and that takes a little bit longer. But I can give you, we asked Very quickly after we started this, can you give us some examples where this is making an impact to our 80 social workers? And they gave us like 15 or so examples and pretty much all of them were around domestic violence, domestic abuse.

And I think my takeaway from this is that they were not able to really identify this. This is a very tricky thing to identify. Yeah. And as a result, Of this insight being from any note that existed in the past six months, these things were getting picked up where they wouldn't have otherwise picked it up.

And so there was one example where there was a 20 to 30 year old female, and she came in with very nonspecific complaints. I think she had a headache. She was not feeling well. She was just, something that would have flown under the radar. They would have probably in the ED, they probably would have given her some Tylenol, some ibuprofen, maybe some fluids.

And all the labs would have been fine, and she would have been sent on her way. And this got triggered, and so the ED social workers, they found the exact note. Very quickly, where it identified that this patient had some issues with domestic violence in the past and had come, essentially to the ED previously where this was elicited.

And so the ED social worker understood the situation, and she was able to use essentially a trauma informed approach. When she went to the patient, which is a specific way of kind of eliciting information backed by literature, and so she was able to go essentially to this patient, elicit more information than she would have if she had gone in without understanding this, and she was able to kind of understand what the situation was and that ED visit was really tied to essentially that domestic abuse kind of instance, and she was able to kind of target specific resources that the patient could take away.

Thank you.

I spent a lot of time asking you questions about the why and the outcomes because that's important. I mean, technology is only as valuable as it's driving movement in those areas. I love what you've done here because one of the most frustrating elements, I think, is knowing that we sit on this trove of data that can help us.

Get the insights to drive the outcomes we want, but it's unstructured to your earlier comment and the inability to quickly decipher that it's just a highly frustrating paradox that we find ourselves in, but I do want to kind of switch gears a little bit to maybe talk a little more about the underlying technology and what you've learned there.

So maybe to start with, to give our listeners a sense. If you could t shirt size the resources needed to do something like this the budget kind of cost to do something like this, small, medium, large, what have you learned through this journey?

Yeah, so just to give you an idea, we had a 2 million budget to solve a couple different use cases.

So this was one of the use cases. And this was the one that we were the most highly successful in with that essentially our ELT wanted us to solve for a few different use cases around voice and natural language processing, voice AI and natural language processing this component of it our team was quite broad that actually worked on this but from the IT standpoint, obviously I was involved to essentially provide kind of the clinical informatics and then navigating kind of The clinical aspect of this our AVP over data analytics was kind of, overall the analytics type team members.

There was really a tiger team of three folks that were critical for this. We had an implementation manager. Who is essentially figuring out how to deploy this and essentially figure out the pipeline. We had a data scientist who essentially took these out of the box queries and basically made it our own.

So we essentially extracted. gold standard data set, ran these queries against it, we actually validated it retrospectively. So actually one of my research coordinators was annotating everything to make sure that, these queries made sense. And so we actually improved the accuracy considerably after we validated things and kind of made some tweaks.

And then there's the Epic Analyst. So the Epic Analyst is the one that's doing that front end integration, making sure that the workflow makes sense, really understanding that. And then we had a few folks from our Enterprise Data Warehouse team making sure the back end kind of integration made sense and, enabling all the back end type work.

So that, that's essentially the technical team. Obviously, we had executive sponsors. We had one of our directors in our case management department who essentially Identify, this is a great team to work with, this is like, a key point where they're extracting social determinants.

There's a key issue where they are taking a long time to do this, and you can make a lot of impact here. The on the ground team members, the ED social workers, were absolutely critical for us to find a pilot use case and to showcase that this technology will actually work. So, if you look at the whole life cycle, a lot of people...

There's really like, three or four key members on the technology team that really kind of, helped us with this. I mean, in terms of pricing, this is like, for a year it's kind of like in the six figures. So if you think about where we're at right now with like Gen AI and, chat GPT 3.

54 Chad, GPT 4 is probably an order of magnitude significantly higher than something like this. Yeah. Obviously, that's a generic general type of AI, and we're still trying to figure out, use cases beyond ambient AI. But this gives you, in my view, this gives you a way to really do those specific use cases, and you could probably do it really quickly and do it with like a very agile team.

As we think about our AI strategy, I think this is going to be a critical component to kind of knock out these very specific things that are tied to decision support and maybe kind of, inverting that workflow from 80 percent chart eval to 20 percent chart eval. Okay,

That's really helpful uh, context.

Thank you. I want to ask something to make sure I understand it right and just to get your perspective. So you described how when you started this, It was kind of pre the emergence of mainstream gen AI. So as that kind of, earth axis tilted on us? Did that change your approach at all? Did it accelerate it?

Can you kind of describe if it changed it? And if not, like, is it going to change how you approach it in the future?

I Think having AI in the public consciousness helps a lot. Right. So, I'm sure with. Every health system, board members are very interested in what's your AI strategy? What's your Gen AI strategy?

Every CIO is being asked this. Anyone that's kind of working in this space is what are you thinking about this? And I think they probably are complementary. So the Gen AI strategy I think is going to be tied to what a lot of vendors are doing that have a lot of resources. So, as an Epic shop, Epic is Putting a lot of time and resources into thinking through how Gen AI kind of integrates with Epic to solve for in-basket or Summarizing notes and all of these things.

These are areas that we're not going to play in. They're probably not going to be interested in how do we extract social determinants and tie this into a ED social worker workflow? So these types of point solutions, having this kind of platform allows us to solve critical problems that come up The Gen AI, I think, is a much bigger, like, how do we solve these bigger problems around efficiency tied to, like, in-basket and summarizing notes and stuff like that.

So I think they're complementary and it's really being in the public consciousness is allowing us to actually drive this. There's more enthusiasm for our, you know for this NLP solution. There's a lot more buy-in from our clinicians, trying to, you know, now when I go and I say, Here, why don't you try to use this tool and think about how you can better integrate this into your workflow.

Everybody is now, kind of sold on the AI concept, so it makes it a little bit easier. Yeah, well

said. I should have asked you this early on. Do you have a name for this tool? Have you like branded it internally? What do you call it?

I mean, we still call it linguamatics. That's the original name because we initially put linguamatics identified.

So it's still linguamatics for us, even though it's now an IQVIA product. Yeah. And I,

Used the word platform a minute ago, which I was thinking about before you even said it. And I love that because again, this is not a point solution for Social workers in the emergency department, like that's a use case that you're applying it to, so if you think about this platform that you've created, What are your future plans to expand it?

What are those additional use cases that are curious

for you? So we're in the thick of this, so we kind of, rested on our laurels for a while for this ED social worker thing, but now we're starting to figure out how do we use this and deploy this elsewhere. So it's a combination of a few things. And one of the things that I just kind of an aside I was kind of combing through The Economist, which is one of my favorite papers to read.

And there was this article that said that growth is not just innovation, it's also diffusion. And so diffusion is a key aspect of this. So we have to think through how do we diffuse this across our system so that people are aware that this is a way to solve problems. So we're essentially setting up a governance structure around this technology, around this specific.

NLP solution. That's kind of across our whole system and a variety of a different clinical quality research stakeholders to be part of this. We've also taken a very specific ED social worker workflow and we've generalized it to just a general column that can be used for inpatient, ED, ambulatory, and we're showcasing that to a broad bunch of stakeholders to say, use this in your workflow.

And then, take a couple weeks, a month or so, then come back to us and tell us how you'd like this to actually be used. And then we can, customize it based on what makes sense for your workflow. Because a primary care workflow is going to be very different from an ED workflow, from a crisis worker in the ED, from mental health, from, say, an anesthesiologist in pain.

So, we just want to kind of put this into their hands so they understand what it is that we have. And then they can come up with... Tweaks and customizations to the SDOH workflow, but also, oh, by the way, can you solve this adjacent problem that we have, like in mental health or we're interested in opiate use issues.

Why don't you help us solve that? So that's kind of how we're thinking about it more broadly.

Great. Okay, two final questions. You mentioned that you're in the process of writing and publishing a paper. Any time frame on when that will be published?

our chief medical officer, chief clinical officer is very interested in that, so we're trying to push the gas on this, so.

We're hoping that we can get the analysis done this month. Vlad is writing on my data scientist to extract all this information and how we're going to extract it, because we may need linguamatics to essentially extract whether we've made an impact in terms of closing care gaps. But my suspicion is we'll probably submit by end of year, early next year.

Okay, great. And last question for those listeners who then are just inspired and excited by what they've heard here this podcast. What advice would you have them for kind of the first steps they might want to take to explore this opportunity?

Again, I mean, there's, there were certain Things that we had as requirements when we went down this path, because we, this was tied to an internal grant, we had to solve something very broad within, two years, which is a very short timeframe when you think about IT related projects.

Yeah. So we came into this as though this was a grant process, and we needed to solve this. And some of the key things that we. And we've touched on is solving for a platform. There's a lot of point solutions everywhere and it's like death from point solutions.

So think about this in terms of platforms. I think making sure that things aren't a black box or explainable is very critical. Owning the process I think is very important. So being able to validate , retrospectively, prospectively, absolutely critical to gain that trust.

I've heard from other organizations that have used more neural net type solutions that sometimes when they push it out to a user and it doesn't make sense. And there's no way to figure out why that's not making sense. The team members on the ground just lose trust from it and then stop using it. So I think trust is such a critical piece that everybody really needs to understand.

And then the final thing is, it's a kind of a research construct, the socio technical kind of model, which is essentially it's people, process, and technology. You have to think about the people and the process first. Technology is just the enabler. So if you think about this as a technology solution first, You're going to fail.

You have to think about the workflow of the ED social worker, what is their process, and how do we integrate this so it's seamless within it. So those are some of the key things. Yeah, great.

My guest today has been Dr. Nirav Shah, who is the Medical Director of Quality Innovation and Clinical Practice Analytics at NorthShore Edward Elmhurst Health.

Dr. Shah, thank you so much for the time and for sharing these insights with our community and congrats on your success to date and look forward to following this going forward.

Thank you, Reed, really appreciate

this.

  📍 I love this show. I love hearing what workers and leaders on the front lines are doing.

And we want to thank our hosts who continue to support the community by developing this great content. If you want to support This Week Health, the best way to do that is to let someone else know that you are listening to the show and clue them into it. We have two channels, as This Week Health Conference and This Week Health Newsroom.

You can check them out today and you can find them wherever you listen to podcasts. You can also find them on our website, thisweekealth. com and subscribe there as well. We also want to thank our show partners, Armis, First Health Advisors, Meditech, Transparent, and YouPerform for investing in our mission to develop the next generation of health leaders.

Thanks for listening. That's all for now.