Three CIOs, one MIT AI course - Governance, ROI, and the Right Use Case| The 229 Podcast with Lisa Johnson & Tamara Havenhill-Jacobs

This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong. **Drex DeFord:** [00:00:00] If you're still paying maintenance fees on legacy systems, you can't shut down, you're not alone. Health systems are drowning in cost and risk just to store old data. Harmony Health IT Migrates that clinical and financial data into a secure archives so you can finally decommission those systems and keep every record compliant and accessible. Find out more@harmonyit.com. **Sarah Richardson:** Hi, I am Sarah Richardson, principal at this Week Health and the 2 2 9 Project and a former Healthcare IT executive. I spent a lot of time in rooms with the leaders shaping this industry at dinners, round tables, and events across the country. And every so often someone says something that stops the whole table. You can feel the room shift and everyone leans in. That's the conversation I wanna keep going. Welcome to the 2 2 9 Project podcast. Let's get into it. We're live from Napa. We are at one of our CIO summits, and we are so grateful to be [00:01:00] together in person. It's not very often that we get to do a podcast in person together. It's not. **Lisa Johnson:** It's awesome. **Sarah Richardson:** It's awesome. And even just being together. And so we recently completed a class from MIT about AI strategy and leadership, and this is really about that experience and the things that we learned, the things that we thought would be true, and how we're applying it in our healthcare environments. Uh, some of the lessons learned, really, really key. And when we all agreed to do this together, I don't know that we realized how much would be involved from a time perspective- ... as well as, uh, the 13 weeks, because there was a one-week break. I did 80% of my class on an airplane, uh, thanks to our, just our travel schedule. What about your timing of when you fit in this coursework to your day-to-day operations? **Lisa Johnson:** Yeah. Like, first of all, if I had known how long it would take or how much effort it would be, I probably wouldn't have done it, so I'm really glad I didn't know. Um, and then, uh, I, I think evenings and weekends. So basically, when I wasn't working, that's what I was doing. [00:02:00] And I think I'm thankful for that because we are so busy, and we were able to focus. So it, it made us focus on something and created the time, and it was okay to be selfish because we had to get it done, and we had basically a goal at the end. And so that was super helpful. **Sarah Richardson:** That's a good way to look at it. **Tamarah Jacobs:** I, now I had looked into a certification or a class like this for about a year before w- I found this one or you found this one and we signed on. And I anticipated, I think I anticipated that I knew what I was getting into. I certainly did not actually know what I was getting into. Um, I did mine sometimes on planes, often, uh, later at night or early, early morning. And, you know, Sundays tend to be my AI days, oddly. I mean, maybe not oddly. Just for the past year or so, that's more so where I dedicate a chunk of time that I'm focused on. And so Sundays ended up being [00:03:00] part of it. **Sarah Richardson:** Which was nice because our assignments came out on Thursday morning and weren't due until the following Thursday morning. Yeah. So we had a chance to take the weekend to get things done, which was helpful. Tell me about the most important mindset shift that you experienced from taking this class and how it's shown up for you personally and professionally. **Lisa Johnson:** So I think for me, there was a lot of, uh, things that just kind of reinforced what I knew. And so because of that, it just built my confidence. And so I would say prior to the class, I think I was a little bit more tentative talking about AI. Um, and I also sort of felt like, "Do I belong talking about AI?" Coming from clinical care long ago, I wasn't a programmer or an analyst, and I felt like, "Do I have a space in this environment?" And then at the end, I realized my leadership style, the way I lead teams, what I know, I am built for this. And so, I think it built my confidence in that area, and I feel [00:04:00] like I know where my space is. **Sarah Richardson:** And it led-- You had a big board presentation right maybe towards the end of our class where it was, should we be using AI? Is there a theological element to the usage based on some of the stuff that came out? How did that presentation unfold and how you were able to show up because of the knowledge you'd gained? **Lisa Johnson:** Yeah, I, I don't think I would've been able to do it the way I did it before the class, so I'm super thankful. And yeah, so I, I come from a Catholic health system, and so the Pope just had come out with, you know, his thoughts on AI. And I think there's some really big things that we wrestled with even in the class on, like, how do you keep the humanity basically in the technology? And especially with us in healthcare, when you think about that care delivery experience, and we talk about, like, the sacred encounter at Providence, and that's basically the space between our caregivers and our patients, especially in those times of greatest need. And so, [00:05:00] that really resonated, saying, "Actually, we're gonna be intentional with the AI, and we're actually going to make sure that when we deploy it and utilize it, it's because we're trying to actually bring that human experience back in." So when you think about, like, when we started implementing EHRs back in early two thousands- Mm-hmm ... we put computers in between patients and providers. And so with AI, what I'm so excited about is hopefully that's gonna go to the backdrop now, and that relationship doesn't have something in between that's a tool or technology. But it's just there, and it's there to assist and help and really aid in, in the complexity of care delivery, but as a partner, um, and not in between. **Sarah Richardson:** You've been doing quite a bit in your organization with AI for a period of time. How did this either elevate or bring things forward more quickly than you had been in the past? **Tamarah Jacobs:** What's interesting is, yes, we've been [00:06:00] doing a fair amount with AI, and last fall had built out an internal platform for agent development deployment- Mm-hmm ... anticipating, you know, areas where we were going. And then personally, I'd been doing a significant amount with it for about a period of... a long period of time, actually. And I think what I was really ex- hoping when I was looking into this is that it would help evolve, our governance structure and help me think about that a little bit differently so that it was, . i- there's no gold standard for governance right now. Yeah. I mean, that's so much about what we talk about, whether it's at a dinner or a summit or in, you know, in just in listening to different podcasts or the education components that we get. There was a lot that was very, uh refreshing to know too, similar to what you said. Like, things that we know are, were [00:07:00] reinforced very often in this course, especially things like the quality of our data being so important- Yeah ... in advance of building on top of it., It's the same thing as that concept of garbage in, garbage out. It's, it's we're not gonna fix the quality of our data directly with the use or the application of AI. So that's helped us to think about that. And in the time period, actually during the course, organizationally there's been ... I feel like it's, there's a, been like this dawning that's occurred, and there has been elevated acceptance, elevated interest. We're doing a lot of work to try to upskill. And, and as we talked about, that's, that's a deficiency and a, a gap that I think b- certainly it can't just be us that are dealing with. So it helped give me, um, some different aspects of thinking coming into that, both [00:08:00] to connect across the organization in terms of the things we're thinking about, but often also to try to think differently about how to break down what could very easily become a very administratively heavy, heavily-governed entity to try to find a way to be agile with piloting or deployment. **Sarah Richardson:** How are you finding that healthcare leaders can discern between the right AI opportunity versus the shiny object, versus the vendor promise, versus the person who comes back from listening to whatever podcast over a, a weekend and looking to deploy something as easily as we know that it can be done? How do you put the right structures in place to make sure it's the right thing for your organization? **Lisa Johnson:** Well, and like, first of all, it's really just human nature to be excited about where we are today. I mean, AI is not new, but there's so much white space, um, that's there now. Um, we have capabilities [00:09:00] we just haven't had before, especially in the clinical spaces, which I'm so excited about. But it has to be grounded, so I talked about intentional AI. And so, um, when you look at basically our strategic direction in our 2030, the things that we're targeting are in direct alignment basically with those priorities. And then you really have to take a step back and evaluate, is AI the best way to innovate in this space? We know what the strengths and weaknesses are of AI, and so you need to also be intentional about what tool basically you take out. Just like you don't, you don't use a hammer for something where you need a wrench. And so,, you have to be intentional about that. So we're not just innovating with AI just to innovate for innovation's sake. It really truly is intentional, grounded in our direction and our priorities, and making sure that it is the right tool for the right thing. **Sarah Richardson:** Yeah, what can it actually fix? I mean, we're already hearing in a room with your peers throughout the morning today of, "Well, can AI fix that?" Well, maybe that [00:10:00] it can, or are there other aspects of our intelligence and capabilities that are better applied? What are you seeing, Tamara? **Tamarah Jacobs:** Uh, it is, it is interesting because it is, it, there's a fair amount of subjectivity still to dentifying or determining if it's right for the organization. We struggle, I think we all collectively struggle with establishing a return on the potential work we're doing. It's easier if you're talking about hard dollars. It's, uh, it's more difficult when you're talking about those qualitative aspects- Mm ... of what you anticipate being the benefits because there's no baseline to transition or translate that quality uplift to a, a metric or a data point that can be incorporated consistently. I think that there's probably a way that we're gonna have to find ourselves to that, to having, to establishing some sort of norm that we can use as a balance to determine that. It- [00:11:00] right now we're applying, you know, a, a fair amount of the same- evaluation components that we do with other projects into the, the work that we're doing, but we're also looking for more discrete output in early phases of it. And we're also working to hold ourselves h- uh, to a higher level of accountability for our own monitoring and auditing to determine if it is actually efficient or effective. And I think what we'll see is that there'll be more discussions that we have about whether or not it makes sense to maintain this as part of our tech stack, or to just move it out of it if it's not necessarily giving us what we're expecting. I think we're all kind of probably, through our rationalization processes dealing with the things that we've implemented, and whether we're m- we're tracking it to see if it's producing what we expected or not. It's probably not, and it, it languishes in our [00:12:00] environment. **Lisa Johnson:** And we have to learn, I think, that flexibility, too. Like in healthcare, I think, we tend to be more risk-averse. Mm-hmm. And, we have to be comfortable, I think, with the ambiguity and also kind of, you know, doing away with something that maybe didn't have the value that we expected. Mm-hmm. And so that, propensity to try and then fail- Yeah ... and be courageous around that. And then also, like when we-- you talked about the ROI and the assessment of like what that ROI might be, you can be kind of in analysis paralysis- You can ... around that. Um, and so sometimes you just have to go with your gut, too, a little bit and, and have the courage to say, "Yeah, we're gonna try it and see what happens." **Sarah Richardson:** Are we pulling anything out? You hear us talk about adding AI governance into the model, and I'm like, we already have enough governance. In fact, our friend Amber often talks about unified governance. In other words, you're governing six or seven different things. This is one more. Do we have the rigor to be able to pull the AI solution that's not proving its value within a year out of [00:13:00] production and determine what might replace it? And does that actually require another level of governance? **Tamarah Jacobs:** Yeah. I think to some degree it probably does. Um but I, and I love the, the idea of unified governance and a very... I mean, she was talking about it this morning, and I've already said to her like, "I wanna talk a little bit more about that," because it's really easy to make this be something that we over-architect unintentionally- **Lisa Johnson:** Yeah **Tamarah Jacobs:** but that easily happens. What we're talking a lot, I'm doing, I'm foc- I'm going to be focusing on doing some education across the organization to help people understand that there are different kinds of AI that we're working with, that, so that categorization of it could be deterministic, it could be generative, it could be agentic, it could be a hybrid where you're orchestrating across this, and each of those have a different risk level that's associated to it. And because we have [00:14:00] to establish the, accountability within the organization to build in that human in the loop, and that auditing and monitoring, that's only going to be able to be done by the operational stakeholder or the business stakeholder in partnership with, te- with our technology team. Each one of those has a different level of governance responsibility, and that's part of what I'm trying to think about right now is- Do you-- Are there ways to not establish multiple work groups that you're assuming are gonna do this, but more so to take a sort of an agile or maybe a more nimble approach to it so that each workflow, agent, initiative that we move into production has a, an assigned team of stakeholders that have the responsibility to report back to a core governance on a regular basis on the performance in that? **Lisa Johnson:** Yeah, the [00:15:00] scaffolding, can be consistent, but I love how you're thinking about it too, because it also depends on the use case or how you're using it. Yeah. Is it for clinical care? Is it for operations? Mm. What is the risk factors? And so when we're thinking about it, we're using the same scaffolding for governance, and, sometimes it needs to go through basically more assessment if it's gonna be integrated in clinical care. And then there's also kind of a fast pass, like, like Disney FastPass, for things that- Yeah ... maybe like, and, you know, some, a third party is delivering that we feel really comfortable with that has less risk. We can kinda put it- Yeah ... through that fast pass, lane, but the scaffolding and the framework is consistent, but there's just different routes basically that it needs to take based on what it is- Yeah and the, and the risk involved. **Sarah Richardson:** There's also a layer of putting it in people's hands and letting them play with it in a safe environment, that sandbox. Yes. I mean, this morning, we have a colleague who went to a major provider and said, "How long would it take to do this?" They said a year [00:16:00] and a half. They built it in two weeks. **Lisa Johnson:** Yeah. Yeah. **Sarah Richardson:** And they just played with it to see how it would work. Now, you can still put in the right data framing, the right security measures, the right auditability, and yet just go play with it. Just go and do it, and educating organizations to feel safe in the usage. The art of the possible can start with building your own personal assistant- Yeah which we saw another one of our colleagues do- Yeah ... who has built a whole- It was awesome ... relational, literally a CRM, which in and of itself is about relationships, have built it internally for stakeholder management, which I remember doing that on a spreadsheet and with, you know, a quadrant diagram and writing people's names of who are the people that are my ardent supporters or my ardent detractors, and how do I manage those aspects. What are you hearing internally? What needs to be overcome- Mm-hmm ... for organizations to feel better about its use? You don't go to the engineering team and say, "I need 10 less of you." And yet, when we see a referrals management tool go into place that can do the work of 10 people with two, what [00:17:00] are you hearing, and how do you overcome those fears? **Lisa Johnson:** So I think the biggest shift is cultural, actually. So when you talked about that solution that was basically born out of a few days instead of a year and a half, I don't think... Uh, so I think from our team's standpoint, we have to give them permission and the right environments and the right scaffolding with the governance, et cetera, to actually feel free to, to play, and they have to feel free to make some mistakes. And so I think that cultural shift needs to change, and then I think the thing that I'm hearing from our care delivery leaders, our nurses, our doctors, um, is, is, is that they're also afraid. And I think really the, the main way to tackle that is to bring them to the table and have them basically experience it with us. Because if they touch it, feel it, start working with it, then they're gonna get more comfortable. That's human nature. And so,, just like it's always should have been, those [00:18:00] clinicians should be at the table with us so that we can really get the right output at the end of the day and achieve the right things together. More and more with AI, that's even more important to do. And then the third thing is just to make sure that you relate it to things in their day-to-day. A lot of people are hearing about AI on the news or on TV or through movies, and they don't realize that it's been in our environment for a long time, and they're using it every day. And so once you start relating it to real things that they're utilizing every day, then it's less scary because it's not this big thing that's gonna come down, right?, And they're like, "Oh, okay. So yeah, it is in my environment." And the last thing that's really important is that it is sort of like your partner in your sidecar, um when you're driving. And so the human is still driving. Um, and so, that relationship and how to basically line that out and figure out what those parameters are and how that could work for good, I think is really important as well. [00:19:00] **Sarah Richardson:** Excellent **Tamarah Jacobs:** I, it's interesting, I just, have wrapped up some slides for b- for the board, and one of the things that I included on it is that for us, organizational capacity, change management, and data quality are some of the biggest potential risks that we have in terms of our ability to move quickly. We, in the environment that I described we built out in the fall, included in that the ability for us to stand up a sandbox area. So to your point, we're working on a path to production that will allow us to open up a sandbox where we can extend some of the tools for coworker and, um, coding to individuals that meet the requirements or are very interested, that they can go in and play and work with something and have it run and be tested. And then, in, as it progresses, if we find that it has the, it has [00:20:00] value and is something that we want to move in or activate, it has a path to production that goes through the oversight, testing, governance, potential rebuild into whatever environment we want that to persevere in, and then move in. So we want to be able to harness the creativity and the knowledge of individuals who are sometimes aren't working in different ways, in partnership with our teams. **Sarah Richardson:** So if we're sitting here three years from now having a conversation about the state of technology and where potentially AI has integrated itself even more thoughtfully into our personal and professional lives, what do you believe we would be talking about? **Lisa Johnson:** Oh. Well, I'll tell you what I hope- The wand, yeah if I had my magic wand. Um, so, uh, so su- I'm super passionate about early detection, um, for cancer 'cause I have a personal story around that. And so what my hope is [00:21:00] is that we continue to just leapfrog leaps and bounds on pathology and imaging, and making sure that we see early detection markers so that we, we have better outcomes, in all kinds of disease states. So that is one thing. And then the second thing I'll just double down on, which is what I mentioned earlier, that we continue to bring the joy back into practice, coming from clinical care myself, and really truly see that the technology is an enabler and it's not coming in between, right? And it's not always taxing or overhead, that it's truly helping the system and care delivery be brighter and better, for everyone that's involved. And so, we, we've gotta get there, and I'm hoping that soon we do **Tamarah Jacobs:** It's interesting. Our, part of our miss- mission is grounded in bringing the joy of practice back in, and I agree with so many things that you said. I think, too, for me, I would [00:22:00] hope and believe that it becomes something that is n- not s- our conversations aren't structured around fear or the challenge in activating it, but that it is, that it's a tool that is in use on a regular basis. We assume that it's a partner for us without having to have difficult conversations sometimes about either the potential it could have or the change it may bring. And that what we're talking about instead is really how it's completely transformed the work we're doing, redesigned that experience for our patients, for our providers, and for our staff, and we're really able to influence just the state of healthcare. **Sarah Richardson:** Agree, because there are two things that could be true from what we're experiencing with AI today, is that [00:23:00] it becomes a true equalizer of access to care, that it removes the socioeconomic burden with certain either cultures or the ability to get care where you need it, whether that's rural or an underrepresented population, that everybody has access to the same type of care. And then it's part of your wellness journey versus your sickness journey, that you can stand on that scale that's got your mirror in front of you, and it can do a scan of you and track long-term hotspots, track things that aren't self-healing in your body, and realize that the ability to have better longevity and a, a healthy, happy life, even independently, is not something that is based on your zip code- Yeah or even your access to Wi-Fi at that point, that it really becomes something that normalizes our access to care across the continuum for sure. And it **Lisa Johnson:** could be truly personalized care. **Sarah Richardson:** And it's 100% about you. **Tamarah Jacobs:** Yeah. **Sarah Richardson:** Yeah. My fav- Everything from your genomic mapping all the way through to the vitamins you decide to take- Yeah and how much water you need to drink tomorrow. **Tamarah Jacobs:** My [00:24:00] favorite agent that I've created or app that I've created is s- is essentially that. **Sarah Richardson:** That tracks my wellness. **Tamarah Jacobs:** And it's got everything. **Sarah Richardson:** Yes. **Tamarah Jacobs:** Yeah. **Sarah Richardson:** Agree. I'm glad we went to MIT- **Tamarah Jacobs:** Yeah ... **Sarah Richardson:** together. I am too. We can say we went to MIT together- Yeah ... even though, uh, that was, it was a 13-week experience that, uh, really did, I believe, give us a different perspective, that you've all said it gave us confidence to have these conversations in bigger rooms that need to hear it. **Tamarah Jacobs:** Yeah. Agreed. **Sarah Richardson:** Thanks for being here, thanks for being in Napa- Yeah ... and thanks for being my late-night classmates. **Lisa Johnson:** Thank you. **Tamarah Jacobs:** Yeah, it was great. Thank you. **Sarah Richardson:** Agree. Thanks for listening. The conversations happening in healthcare it right now are too important to stay in the room, and that's exactly why we bring them here. If today's episode made you think, share it with someone who needs to hear it. Subscribe to the 2 29 Project podcast at this week, health.com/subscribe and come find us at the next event 2 29 project.com. These conversations are better when more of us are in them. See you next time. [00:25:00]