March 25, 2020: For this episode in our COVID-19 series, we are joined by Josh Peacock and Kyle Quinby from Sirius Healthcare to talk about scaling VDI and clinical systems. We are shifting from understanding the problem set to more solution focussed content and Kyle and Josh have done some great work in addressing how to scale systems during a time of crisis. This episode is a pragmatic discussion on this process at present and in the coming weeks, as we look at connectivity in the cloud and allowing more people in health sector to work remotely. We get into the ins and outs of this transition and the steps that Sirius has taken to allow enough runway for these changes. With so many potential hurdles and potholes, it is great to see such dedicated minds applied to the current crisis! Listeners will hear about how they are creating more bandwidth, safeguarding against the dangers of VPN connectivity and leveraging the existing systems into something workable for current needs. We finish off the chat talking about the great resources that our guests and Sirius have made available and where to find them!
Key Points From This Episode:
Coronavirus Series: Scaling Citrix Rapidly with John Shelsta
Episode 208: Transcript - March 25, 2020
This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.
[0:00:04.8] BR: Welcome to This Week in Health IT with our continuing series on coronavirus prep. We’ve moved from understanding the health IT problems that the health systems are facing, to looking at a set of solutions.
My name is Bill Russell, healthcare, CIO, coach and creator of This Week in Health IT, a set of podcast videos and collaboration events, dedicated to developing the next generation of health leaders.
Before we get going, I want to give you three resources for those in health IT during the crisis. For CIOs, I’m currently helping my coaching clients walk through this as a sounding board and advisor through this time. If you’d like to connect with me over the next two months to experience at no cost, shoot me a note at [email protected], or you can hit the website healthlyrics.com.
For anyone in health IT, two more resources; for anyone in health IT who finds themselves in uncharted territory and you want to crowdsource solutions with a team of seasoned professionals, as well as other health systems, we've set up two ways to do that. We now have a Slack channel that is monitored and supported by our sponsors who I am greatly appreciative of how they have stepped up; VMware, Galen Healthcare, Sirius Healthcare, Starbridge Advisors and Pro Talent Advisors, as well as Health Lyrics. I'm monitoring the channel as well.
The Slack channel allows you to get your questions answered. Things like can my telehealth scale, we're having problems with VDI, what's the best cell health solution? What's a good backup telehealth solution? Which ones are using for clinical versus business? You name it. That channel is designed for you. We launched that on Tuesday, yesterday. We have gotten all the technical resources on it and the channel, the glitches worked out, so we are good to go.
If you want access to that, send a note to Slack at thisweekinhealthit.com and we will get you set up immediately. You're going to have to use your health system e-mail address in order to do that. That's the only way we can verify that you're with a health system. We want to protect the integrity of the channel.
Our sponsors are also supporting an e-mail channel at [email protected] and you can tell we're moving a little fast here, but it's [email protected]. If you just want to shoot an e-mail over, get it routed to an expert in your area from our group of sponsors, we will do that for you as well. Again, I want to thank each of our sponsors for stepping up to provide these services to our listeners and the industry.
Now we're going on to another – actually, one of the first of a series of interviews I'm going to be doing with technologists around what they are finding as they work with their clients and what solutions they are finding that are working. Today I'm talking to two people from Sirius Healthcare. I’m talking to Josh Peacock, who is the Health Solutions Architect for four Sirius, as well as Kyle Quinby.
The two of them have been working on solutions to scale VDI and to scale the clinical settings within health systems and they have written some really good material and I'm looking forward to sharing that with you.
[0:03:05.2] BR: All right, we're joined today with Josh Peacock, the Health Solution Architect for Sirius, and Kyle Quinby, a Infrastructure Solutions Architect as well. I'm really excited about this conversation. We've transitioned the conversation from really understanding the problem set, so we've talked to a lot of CIOs and they're saying things like, scale my VDI environment. How do I ramp up people working from all over the place, telehealth? Obviously, people are ramping that up and whatnot.
What I wanted to do was talk to people who are solving some of these problems and get really pragmatic and into the weeds. You two, they locked the two of you in a room and said, “Figure these things out.” I love the fact that they gave me access to the two of you. What are the things – Josh, we’ll start with you. What are the things that you're working on in the lab right now for your clients?
[0:03:58.6] JP: Some of it, we really took what we learned, even what we talked about in the last podcast. I was on with you around cloud connectivity and using the cloud as a way to use VR. We took that, modified it a little bit and offering that up as a way to expand and burst certain components of the environment up in the cloud in a pretty short timeframe. The other one which Kyle can talk about, he's been dealing with a lot is really a remote workforce and how do we help enable that group to do stuff securely and effectively and from home.
[0:04:28.9] BR: Awesome. All right, so let's break this down. We'll start with VDI. We just sent a whole bunch of people home. Actually, we sent most of the workers home that we possibly could. I'm hearing people say scaling this stuff up, in order of magnitude, which is just up until this point unheard of. We would have never done a project this quickly. Kyle, what are some of the considerations as people are ramping up this VDI environment, sending people home, getting them situated? What are the problems that you guys are looking at and trying to solve?
[0:05:09.0] KQ: Sure. A couple. Number one is while we are looking at capacity, security is a huge concern. We’re going home on the family PC that my kids have downloaded who knows what to and how to make sure that I can use that to get into an environment and keep things secure. We've been looking to leverage a lot of the existing VDI solutions and vendors that we know and love.
We see a lot of customers trying to leverage VPN, which obviously can be a little bit risky, to just connect that home PC up to the corporate network. We're really looking at these VDI solutions that can be allowed as endpoints, those unsecured endpoints to get in securely. At the same time, it's really got to be easy for the end users. It cannot be a lot of steps, because at this point we just don't have the time to give everybody the white glove treatment. We have got to be able to get people in really fast.
For example, I just got off with a customer right before this this talk that normally had a high watermark, about 450 users and yesterday had 4,500 users on their system. That ramping, just you've got to be able to scale it really, really quickly and it's got to be dead simple to get in place.
[0:06:34.0] BR: Yeah. Oh, my gosh. I have a thousand questions now, so we'll try to make this a shorter podcast. My first question is so you're connecting up a VPN, which would be the first thing that somebody would think, “Oh, well we'll just connect up the VPN.” The problem is you have no idea where – it's like, you have no idea where that computer's been. You just can't connect it up.
You want to create an abstraction layer. This abstraction layer is delivering that VDI workstation down. Are you simplifying the application mix of what you're actually delivering down, or you trying to deliver a core desktop that has been pre thought-out prior to this?
[0:07:18.6] KQ: Well, I mean, this is something Josh and I have been working with. I think right now, we're looking at the minimum amount of applications to get the job done, right? The less applications we have in the mix, the easier the challenge is. A lot of what we've been focusing on is the core EMR apps that are needed to work effectively. At the end of the day, we're just trying to support the folks that are on the frontlines trying to take care of this.
[0:07:41.7] BR: The problem with VDI – so the challenge with VDI is you go from these workstations running all the applications, which is really simple when you think about it. Everybody goes home and works on their own workstation, but essentially what you're doing is running the workstation in the data center as all of our listeners would understand. How are you ramping up the equipment? Because most organizations weren't scaled up for 4,500 workstations prior to this. They were scaled to whatever their run rate was, which was plus or minus 20%. They were probably scaled to 600 workstations. Now all of a sudden, they're at 4,500. How do they burst that up? How do they ramp that up so quickly in the data center?
[0:08:26.6] KQ: Yeah, so we're air-dropping servers. No. We can’t do that. Some hospitals, or some locations have additional capacity in their DR. In some cases, they're required to by standards. In some cases, they're able to flex, where let's take advantage of our DR capacity. In some cases, that's enough to meet the demand. In some cases it's not, or they don't have that capacity in DR.
At this point, a lot of hospitals are asking all their workers to come in. They're just absolutely swamped. That's where the cloud comes in, the public cloud. Both Azure and AWS were able to take advantage of that's the whole thing about public cloud, is to be able to flex and expand your capacity on demand, so this is almost textbook how cloud suits the problem.
In a lot of cases, what we're doing is we're hooking up via VPN tunnel, or existing connectivity options that they already have a presence in public cloud and connecting virtual networks where we can spin these desktops up in the cloud and be able to traverse back on-prem securely. Again, securely, we want to make sure we're not just opening things up, or exposing records. We're trying to do this as securely as possible.
[0:09:45.7] BR: One of the things I like is you guys sent me over some documents prior to this. Josh, eventually I'm going to ask you for where people can find these documents, so you're going to want to have the URL handy. I'm looking at these documents and you made it so simple that a CIO can understand them. It's almost like a run book, or a recipe book for this is how you do it. This is how you're going to scale out. This is how you're going to scale up in the cloud. That's stuff we need in a crisis. You don't need these just really challenging, difficult things to scale up.
All right. Kyle, is there anything I forgot to ask you? Because again, I'm a CIO. I'm not as technical as you, that you guys are focusing on that I should be – that I should have asked that I'm missing?
[0:10:35.9] KQ: I don't think so. I think you nailed it. It’s ease-of-use, security and making sure that we're focused on the base apps that are needed to get the job done. That's the way to get it done as quickly as humanly possible.
[0:10:51.5] BR: I'm sitting here on my Mac. You’re going to be able to let me run things on my Mac in this environment?
[0:10:57.5] KQ: Yeah. Mac, Windows, Linux. There's pretty much endpoints, or software for any number of endpoint devices.
[0:11:06.1] BR: Because when you send that many people home, you really don't have control over what's going to be used. Josh, let's talk about scaling up these core applications. Obviously, the most pressing one is the EHR. I imagine a whole bunch of use cases just materialized overnight that people had not anticipated. Potentially, number of users grew, potentially the workload on those systems grew. Is that what you're finding? Is that what you're hearing from your clients? What challenges are they facing?
[0:11:40.5] JP: Definitely an exploration to make sure that they understand what their options are. One of things we've seen is actually because of people staying home, cancelling of electives and other things like that, it's actually reduced the amount of consumption of some of our clients’ environments. We're also expecting as some of these really hard-hit areas have to ramp up and increase bed counts and maybe we're hearing that they're going to be looking to bring back retired clinicians and other people from private practice and such.
We do expect that there might be an increase. What we were trying to do is just provide some options, so that if they get into a situation for whatever reason, they don't have the opportunity to expand internally and they need that extra capacity that we can provide something from a delivery of the EHR core application.
[0:12:29.2] BR: If you came into my environment, let's just take the — assume I am an Epic shop. I'm ramped up and all of a sudden, my utilization started to peak or go over. Is that the thing we're talking about at this point?
[0:12:44.9] JP: Yeah. We obviously like to have a little bit of ramp time to help you alongside that as you're seeing. Some of the things that even ahead of time that you could be looking into to request, for example, ISPs. If you don't already have express routes, or direct connects, or whatever to your preferred cloud vendors that are closed, asking your ISPs if there's a way that they can quickly turn up and give you more bandwidth to be able to take care of the need down the road, that provides some quick steps to help us start prepare for it. Then the team can come in and help address building out that environment in the cloud to scale up and to burst into.
[0:13:24.2] BR: Again, if I'm an Epic shop, most of my stuff is on-prem. I mean, I know there's a bunch of hosts there, but there's a lot of on-prem. Kyle talked about utilizing DR capabilities. Is that what you guys – you're tapping into that stuff in this –
[0:13:40.5] JP: Sorry. Part of what we would also see to – so our team, like Kyle is super busy helping organizations inside and outside of the healthcare industry right now. The healthcare team has really been taking it to focus on addressing the situation initially to see what we can help out with. One of those would be if there's an organization, they're already active-active. They likely have 200% of capability between sites. If they're active passive for Citrix in their environment, we can maybe help them spin up if there's not too much latency between.
There's a lot of things that we could help address before we even have to get the cloud, but the cloud does offer up as an option at the end if we can't utilize stuff they have on the ground. Maybe even that's – Kyle and I have been talking about how do we segment users, so that maybe more business focused users have the more degraded performance environment. If you're too far away or a lot of latency, they can use that because they can handle that extra degradation in performance. And then focus the clinical teams that have the stuff that's on-prem and most valuable performance.
[0:14:42.2] BR: We're talking about the EHR. What other systems are you guys looking at? What other clinical systems are you looking at that may see increased capacity as we step into this?
[0:14:55.8] JP: Yes. We've been looking at a lot of – mostly, the business side apps have been potentially VDI options and some of that's new exploration for organizations, because they haven't had true VDI in the past. Most of it has been focused currently on EHRs, just because that is what we have. It's the easiest and most known quantity that we can expand up into the cloud at the time.
[0:15:19.1] BR: You know what's crazy? We were doing VDI and we stood up our first — we chose a small location and we stood it up. We learned a lot of things. Virtualizing those applications, packaging them up, that whole thing, I think took about three and a half, four months to really get that desktop solid. Then we started to work on scaling it. I would say it took us some good six months or so to scale it effectively; a lot of things about latency, a lot of things about bandwidth and latency essentially, as we were progressing. We're doing that very rapidly now.
How long does it take from the point that a client calls you and says, “Look, we need to scale up our EHR. We need to scale up our VDI and we'll keep those two separate.” How long does it take for them to get from where they're at to where they're going, assuming they probably have some equipment they can scale to, they probably have some DR they can scale to, but they may even need to go to a third tier, which is off-site and those things. Let's just start the question there, which is how long does this usually take?
[0:16:38.7] JP: I want to add one point quick too is that we've been focused in on leveraging technology they already have in place, because we don't want to put any extra load on the helpdesk to the teams who’ve been trying to support the stuff too. We're leveraging technology we already have in place as much as possible in the conversations. As far as deployment and those pieces, I'll actually let iOS speak to those.
[0:17:00.6] KQ: Yeah. Basically, what we put together in the documentation that you were taking a look at is all about speed, because we don't have time to waste here. It is leveraging existing systems. Just to use Citrix as an example, we're treating Azure or AWS as another resource location. Most of the work is taking the image that they already have that's been vetted, so you talked about taking several months to get an image really solid, to where you liked it and you can roll out to your users. We're not trying to reinvent the wheel there.
In a lot of cases, we can take the image that they already have on-prem and prep it for Azure or AWS. There's a documented process on how to get that ready. Then we can put it out there and essentially spin up just a stamp out a whole bunch more hosts and a whole bunch of more resources using that image. In that sense, it's already pre-vetted, it's already pre-baked. We're just extending what they already had and eventually cloud bursting. Taking what they have and just giving them a lot more room, because they can't get additional servers shipped in.
We're giving them the compute cycles. We're giving them the memory cycles. We’re giving them the disks and we're taking their image and just running it up there and just making sure we can back all that traffic over the network link. It's actually something where it's very short runway to go from, “Oh, my gosh, I need help,” to, "Okay, we got our resources up and running. We've got additional users. We got additional capacity in the system."
[0:18:34.3] BR: Yeah. I love that. That is just the scene from Apollo 13, isn't it? They locked you in a room, they dumped all the stuff out and said, “Okay, we're not going to spin up new this, we're not going to spin up new this.” With the equipment that's on site as quickly as possible, how do we go from 450 users to 4,500 users and that's the challenge.
Josh, where do they go to find these resources?
[0:18:58.0] KQ: Yeah. We'll be posting. Sirius has a site that we're supporting across the industry, but we'll have an area that's specific to healthcare around COVID-19, so we'll have that out and available. We're going to link to you to you Bill to post along with it.
[0:19:11.4] BR: All right. I will download those and put them up on the This Week in Health IT website. We'll put it under the COVID-19 resources. You guys can get those things. Then if they want to contact you guys, they can do that as well. What's the best way to get in touch with you guys, or is there a better way to get in touch with you than just going direct? I would assume there is.
[0:19:39.1] JP: I'll take a direct call. Anybody on the healthcare team also and on your Slack channel that's opening up. We'll have that a couple of staff on that to help out to answer your questions too.
[0:19:47.6] BR: Yeah, I appreciate that. You guys are manning the Slack channel as well. If you guys want access to these guys to hit them with a couple of questions and get some support, that's another way to do that. Guys, thanks for taking the time. I know you're busy and I appreciate the work that you're doing for the industry right now. I look forward – are you in the process of thinking through another white paper that you're going to be writing?
[0:20:15.4] JP: We do have other ones that are in-flight being written up about different aspects. A lot of it about how we support different access methods, as maybe we expand some of the testing and remote clinic capabilities as well too.
[0:20:31.6] BR: Yeah. I don't want to give people the wrong impression, white paper. The thing I loved about it was it wasn't the normal white paper jargon. It was really right to the point, step-by-step, here's how you get from point A to point B. I really appreciate you guys taking the time.
[0:20:47.9] KQ: Thanks.
[END OF EPISODE]
[0:20:48.4] BR: That's all for this week. Special thanks to our sponsors VMware, Starbridge Advisors, Galen Healthcare, Health Lyrics and Pro Talent Advisors for choosing to invest in developing the next generation of health leaders.
This shows a production of This Week in Health IT. For more great content, you can check out our website thisweekhealth.com, or the YouTube channel. If you want to support the show, best way to do that share it with a peer, however you do that, send them an e-mail, send them a whatever, DM, whatever you do, that's what you should do.
We'll be back again with more shows as this progresses. We'll try to drop one a day until we flatten the curve. Thanks for listening. That's all for now.