This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.

Today on This Week Health.

if Cinnamon at Erlanger has workflow, that takes 33 steps to do and Yale has one that's 43 steps.

We now have an opportunity to help our clients optimize and be even more efficient

Thanks for joining us on this keynote episode, a this week health conference show. My name is Bill Russell. I'm a former CIO for a 16 hospital system and creator of this week Health, A set of channels dedicated to keeping health IT staff current and engaged. For five years, we've been making podcasts that amplify great thinking to propel healthcare forward. Now onto our show.

Hi, I'm Sue Shade, principal at Starbridge and. and one of the town hall moderators. Today, I'm doing a solution showcase with SureTest and I am joined by Laura O'Toole, CEO of SureTest and Cinnamon Mathews, the Chief Application Officer at Erlinger Health. So, welcome. This is going to be

fun.

Thank

you. Yeah, great to see you again.

So I hope we're going to learn about

the problem that you had to solve and the solution that you provided in the whole area of automated testing great.

Terrific. Okay, so let's start with you, Cinnamon. Okay? Can you give us an idea of the problem that your health system was trying to solve?

Absolutely. So, as we've heard at the conference and just in the industries it's just very hard to be fully staffed and have enough people to do what you've got to achieve for your health system. And we were running into that, and as you start to step through, how can we free up resources, a big 📍 portion of time is spent on testing

As in regression

testing

every month.

Or unit, or smoke, or parallel unit. SUs,

every, all the changes that come in, all the upgrades. And so,

I was just very excited from 📍 the outset . The big difference in what I previously had is that we had to do it from scratch. SureTest just comes with their library that is just so impressive.

Great, great. Let's hear a little bit about the problem in the industry from your perspective, Laura.

How, how prevalent is this issue?

I mean, I think it's more prevalent than people even realize that they have the problem. So as Cinnamon said her staff who are vying for attention. They've got multiple projects and competing projects that they're working on. And they're spending a typical health system like Erlanger, especially those that use Epic and Cerner, are spending somewhere between 15 and 25, 000 hours a year manually testing and regression testing and testing all the SUs.

So

15 to how much?

15 to 25, 000. So 8 to 12 FTEs per year.

That's where I was going. Okay. Thank you.

8, 10, 12. And, we don't tell our clients you can eliminate those positions, but what our clients, because they need them, right? They don't have enough. But what our clients have been able to do is to repurpose them to work on more strategic projects.

And quite frankly, I think Dave said, your CIO said, I want my people to be working at their top capability and solving problems for our customers. I don't want them down worrying about manual testing. It's certainly not a staff satisfier in any way. So it's a problem across the industry.

We saw that it was a problem. A lot of clients try to implement automation and test automation on their own, and they'll buy tools. But it's one thing to get a tool. It's another to have the dedicated resources to not only develop the scripts, but then it's like painting the Golden Gate Bridge, because you have to continually maintain them.

And that was a problem we saw in the market. Our clients are smart people. They could do some development themselves, but what they couldn't do was keep up with the community. Continual change that comes with the EHR, and that's where we come in, and we bring a library of all our clients, a global library of all the workflows that our clients have, leverage that to create a personalized library for our, in this instance, for our library.

That in itself is huge. Yep. That you're bringing that library based on your experience with other clients into each client, right? Exactly. So, can we go back to the staffing, which you started with sentiment as the big issue? And Laura, you said eight to 12. Just, typically. The equivalent of those.

Okay. But if we stick with that or maybe use the example how many you had after you deployed Suretest. And I'm assuming you can tell me more about it. You deployed it fully. So after you deployed, what difference did it make in your staffing?

We've been doing it in phases and we've completed the first phase where we, we concentrated on our regression end to end scripts.

So what that. Every time we did an upgrade I insisted that we test one of each patient type. Just to make sure that we could see a patient in the ED. We could see a patient all the way through surgery. We could have a inpatient, clinic. Because that, that gave us that level of confidence that the upgrade didn't mess up the major workflows with handling, taking care of our patients.

And so, just doing those five regression 📍 scripts from one upgrade to the next, it was 291. That we saved just from what it took to do our upgrade in May, compared to what it was before in February. I mean, and that is just an enormous amount of time. And now, then we turned around and we did our, our hyperdrive upgrade that Epic's doing, and they're moving everybody to hyperdrive from hyperspace.

That would have taken a long time. If we had developed those scripts by ourselves, we'd have been starting all over again from scratch. And with the SureTest had already done some of those implementations for other sites. They were actually able to help guide us through how we needed to adjust our scripts.

So not only did we not start over, we started down the road quite a ways from their experience that they'd already had.

I got a question about the library. Yeah.

based on what you did, are you helping to build the library that now other clients

will benefit from? Absolutely.

And that's always been my dream for the company.

We could have this global library and eventually our clients could share in this value. And it's a real optimization thing. So, right now we're beginning to look at analytics and machine learning where if Cinnamon at Erlanger has workflow, that takes 33 steps to do and Yale has one that's 43 steps.

We now have an opportunity to help our clients optimize and be even more efficient. So... The global library is shared, but each client has their own de identified, personalized library. So, obviously we protect that, that workflow G2 that they have. But it certainly is a jumping off point to give speed to value, which is why we can get the automation deployed so quickly.

Great. So what obstacles did you run into in deployment and in your whole solution journey?

The most interesting one... When I found out about what SureTest offers and the fact that I've had previous experience with my own script developer, I couldn't wait. But my team were very afraid. They were very afraid to let go of control of testing.

Because they take such a high level of pride in personally assuring that everything is working and good to go. And so when we introduced it and said, hey, here's what we want to do, we got a lot of, oh, I don't know, pushback. I'm just not so sure I'm ready to have that happen. But as they went through the process and worked with the SureTest team, their confidence grew.

And then, like I said, when we got to that first upgrade where they would have spent a solid week doing regression testing, and it just ran, they were sold. And now they're like, they're so excited. They're like, what can we do next? they saw the results that convinced them.

And it's like anything.

You have an analyst or a manager that numbers... Automation is going to replace what you do. I think it's natural to be afraid. What does this mean for my job? And I think it's really important that we as leaders help people understand we're trying to make your job better for you. We're trying to free up time so you can do more strategic things.

It's not about displacing you. We have had some clients be able to because of attrition, not refill positions. But our clients typically don't take the approach of eliminating staff. It's just to let them work

more strategically. And what you're articulating, Laura, is certainly people's fear of their jobs relative to robots or AI or

any partners like that.

I think Laura's exactly right. It's more meaningful work that they get to do. And it's more high profile projects that have a more direct benefit. Because if they're not spending a week testing what they've built, they can just review results, they can move on to building the next great thing we need.

Right, exactly right. That's exactly right.

In terms of what your process was for evaluating solutions and, not naming competitors, but just what did you go through to to decide on SureTest?

Yeah, absolutely. So,

there was three options. One was to go the route I've gone before, which is to have my own developer.

The second was, other competitors out there. And then, of course, the third was SureTest. And for us, the selling factor was that library and the archive. And... The fact that we could benefit from all the work that they've invested so far. And not have to start over. And then I'll say, the other thing is, as we've gone along, is it's the partnership and the relationship.

It's the fact that they're personally invested in helping us get to the next best place. They partner very well with my engineer who runs the testing. We're implementing Beaker right now with Epic. Okay. And we've already had calls about how to get that into our scripts. And, again, with that...

SureTest comes and says, here's what we've seen other sites do and here's what we recommend, which just really helps because we're still learning ourselves as we're implementing. And so it's just been a great partnership.

And I don't, I hate being called a vendor.

I want to be a partner. It can sound interesting to me if I'm just a vendor. It really is about a partnership. I believe, I believe in win win. I really do. And I think the other reason I think why we won out is because there's nobody else right now that... It's treating this and doing this as a managed service.

So, that's the difference. You might have some staff augmentation come in to help you get some development done. Then they go away, and it sits on a shelf, and it gets old. Now you're painting the Golden Gate Bridge. That's essentially what you have to do with these scripts. We take full responsibility to keep those scripts updated with every Nova Note for Epic.

There's an example that they take every upgrade. Everything that they're taking, it's our responsibility to get those scripts ready for that next SU, or that next upgrade. We want their updates. Because testing becomes a part of the whole process. Got it.

So more than the tool, it's staff involved. From your team.

Oh yeah, we have a whole team of developers. We have all full time employees. I think the other thing that's a real advantage is many of them were certified, Epic builders and they were consultants as well as knowing the development tools for automation. And so they're able to see through the muck to the other side pretty quickly.

Weekly 'cause they, they truly understand the workflow.

Okay.

Alright. So we've heard about your experience. Do you want to, Laura, describe any other client experience, some of the challenges they've had with deploying this and how they've addressed

them? Yeah. I think the clients that have been the most successful are the ones that have engaged their analysts in managers early on, even in the sales process.

So that we take away that fear component. Like, what is this gonna, that you talked about, right? What is this gonna do to my job? Really made them a part of the solution. And got them to see early on in the process that this is going to give you back thousands of hours over time. I think those are the clients that have been the most successful.

And our clients that have said, like Cinema is doing, we're going to start out with our ambulatory or our inpatient scripts and then we're going to evolve. Because this is an enterprise tool. So all third party integrated applications, Workday, whatever, whatever Oracle, PeopleSoft, we're looking at a people.

So it's an enterprise level tool. So I think when people embrace it and really understand what they're going to get back and they become a part of the process those are our clients that, have been, have been the most successful. Do you,

do you find that clients, I think what you described, Cinnamon, is a pretty phased approach?

Yes. Do you, is that what you recommend, Laura, and that what you find clients doing is more phased over time?

Yeah, so

what we do, everything is done virtually. So we're essentially recording the workflow. And understanding the workflow. And then we go off and do our development and then we send back the automation working in real time so it can be validated.

So yes, typically wherever our clients are having the biggest pain point is where we start. A lot of times that's on the ambulatory side because there's so much disparate happenings amongst the clinics. As in disparate workflows? Yes, exactly, exactly. So just trying to see where you can get alignment in those scripts.

Yes, because our approach to testing is I liken it to learning. So our library and our framework is built so that when we make one update, it goes through the whole library and it finds that component and it updates that portion of the script. So you want to get the broadest coverage with the least amount of crossover and workflow that you can get.

So where the workflows cross over is where we tend, where we tend to start to get, really out of the gate, big bang for the, big bang.

You talked about EHR, right? Great, but do you have clients who have gone who are using it for EHR, ERP, DRM,

all their major systems?

Yes, so we have clients that, most of our clients we started, our library was built we have an Epic and a Cerner library. That's where we started our library. Since then though, all the third party applications, so a lot of the lab systems disparate pharmacy systems tons of third party applications that we have now, I don't even know how many, that we've added to our library.

Library based on our client's needs, we're about to do our first workday. So we're excited about that. And then we just had a client come to us they're on PeopleSoft and they wanna, they wanna move forward with PeopleSoft at the same time as their, as their EHR. Okay. So, I think the clients that are embracing it, we're, we're thrilled.

'cause all of our clients have expanded for our projects, our three year projects, and our first three original clients have all renewed. And now we, we have 20. It's grown pretty significantly, and we expect to add another 20 or so through this year. And that's where I think the real value for our clients.

It just, it just, Because the library's building. Exactly, the library's building. It's bigger, and they're a part of that library. Right. They're a part of that community. Alright.

So, Cinnamon, what are some of the other positive results you've seen from this?

Besides, obviously the staff savings in time it's, it's watching them go from being somewhat fearful to now and what they've created and what it brings back to them every month.

And then I think the future opportunities, I think even just sitting here listening to Laura talk, I'm like, we have people soft. Yeah, exactly. Ding, ding,

ding. And now with our Cerner clients, we're doing a lot of RPA use cases. And Epic is starting to loosen that up. We were talking about that today.

So we'll see where that goes on the Epic side. But, I mean, this is an RPA, tool. It's in our library. We have Use cases in our libraries for credentialing positions. Every time you get a new employee, I mean, imagine if all that data entry can be automated. So I think it's really unlimited, in terms of what the solution can do.

And we're just excited to continue to build, and we're thrilled with the growth that we've seen. We've just celebrated our kind of official really our third year anniversary. But it's really only been two years that, we spent the first year building out our library because... I felt very strongly, and as did the rest of our leadership, we did not want to go out and deliver something to the market we could not deliver, and that didn't have quality.

It was very important to us. So we've been very purposeful in our growth, because we want to do it right, and we want to renew all our clients.

That's what

any IT leader wants to hear, as a partner, as opposed to a vendor. Right,

exactly right. It was very important to us, and we took our time and really built out the framework, because when we go to our client, we say, we're bringing you this library, and...

And there's speed to value. What would take you three years internally to build, we can do in six months, and now we're going to maintain it for you. I mean what I say and I say what I mean. I'm not going to go out to the market and say that I can deliver something that I can't. So, we're just thrilled.

We're thrilled with how the solution's done and, the market recognition it's beginning to get.

Excellent. So, for anybody watching or listening to this who's going, oh, I've got to deal with this. I've got to deal with my testing issues and find a solution. How did you make the case internally to make this investment?

How'd you get the funding?

We did an ROI study,

right? We did, we did. An ROI? Yes.

Tell us about that without

giving us numbers. Yeah, we put together a business case with the ROI and we were able to show, because, our backlog of projects, like most healthcare organizations, is just, I don't know if you had enough staff to ever get it all done.

Right. So we were able to show how much. We were able to equate that to money. anD then freeing those resources up, we were able to tie that into if we brought in external resources to help us, that it would cost X amount. So just through all of those, I mean, it was just, it was a very, I won't say it was an easy sell.

I was going to say, was it a hard sell? I would say that it was, it was not difficult, but we just had to have all of our facts. Make sure that... And since then, we've been, the nicest part is, we've been able to go back to the ROI and even exceed what we promised. Oh, well that's a good news story.

Absolutely. I mean, I was

thrilled when someone from Cinnamon's team just out of the blue called us and said, they... It reduced the amount of time that they were spending manually testing by 98 percent per share test. And they called us and told us that. I'm like, can we make that a case study? Yeah, and they've been, it's so awesome when you see the light bulb go off and people say, I can trust this.

I don't have to do it. And I can go work on something that's, quite frankly, I enjoy more than doing testing. Yeah.

So is

the experience that Cinnamon described with. Getting the support and the, and the ROI and selling internally are common for your

clients?

Yeah, I think it's gotten a lot easier now because we've done so many ROI studies that we have averages now based on the number of bids that you have and the number of employees you have.

We have a whole ROI tool that they can plug in their specific numbers. And the solution, it's a three year solution, and you're getting 100 percent ROI by the end of year one. Alright,

so if I was a CIO and I was interested in pursuing this and talking to you, you'd have this ROI tool? Ready for me. Ready for you to, to start

making my case exactly right.

Well, that, most CIOs I know right now need to Absolutely do that kind of justification. Yeah, exactly. And analysis on, on new

investments. Right, right.

I think when you asked me, the clients that are most successful, it's people like Cinnamon, who are willing to take risk.

And take risk on a new solution. And I encourage you to call every one of our clients. And I encourage our clients to get references. Because if your clients, your clients are your reference. If you do what I said. But what I really respect about the leaders that are choosing this solution is, is three things.

Number one, they're innovative. They're comfortable. Talking about the hard conversation with their staff, which is you have got to relinquish some control. So let me, help me help you. I think Cinnamon does that extraordinarily well. And just the level of engagement and openness to be able to take the solution and continue it to the next level.

It's the innovative leaders that are making the choice to sit back and say, how do I retain my staff? How do I keep attracting the best talent? We have a client that is using this as a recruitment tool. They basically say, you're a certified Epic Builder, you want to come here and build and be out optimizing with your customers, guess what, we have the solution, so you build for me and you need to ask.

They're

using it as a recruiting tool. How's it going? Good, it's going

great. People love it, huh? Yeah, yeah. Okay. Great. But, I just want to give kudos for the innovation and the willingness. Well,

and you're right. I mean, my team used to spend a week each month testing. they weren't like, full on the whole time of that week, but they had to be readily available.

So, it's been a. Because their turn in the script came up. There was no delay in them executing their steps. And now you just turn it on and it just goes to town. It just goes. It just happens. Like

magic. It is. Isn't it

all magic?

Well, it is. I mean, in every single keystroke and mouse click is captured.

So you have a full audit trail. Okay. And it takes screenshots. We've been able to actually find a couple things, I don't know at Erlanger, but in production, that now you can send to Epic and you can reproduce it because it takes a screenshot of everything.

can Show, here's the testing.

Better even when my people

did it. Excellent. Great. Well, it sounds like a great partnership. Thank you both so much. We like each other too, so

it really helped. Okay, there you go. Alright,

thank you. Thank you.

I love the chance to have these conversations. I think If I were a CIO today, I would have every team member listen to a show like this one. I believe it's conference level value every week. If you wanna support this week health, tell someone about our channels that would really benefit us. We have a mission of getting our content into as many hands as possible, and if you're listening to it, hopefully you find value and if you could tell somebody else about it, it helps us to achieve our mission. We have two channels. We have the conference channel, which you're listening. And this week, health Newsroom. Check them out today. You can find them wherever you listen to podcasts. Apple, Google, 📍 overcast. You get the picture. We are everywhere. . Thanks for listening. That's all for now.