Show Notes
How do you train people to talk to other people? For some professions, like nursing, communicating with others is a critical part of the job. Computer scientist Ben Lok developed software for nursing students to interview and examine a “virtual patient.” The patients speak like actual people, have backstories, and don’t necessarily share information easily. The son of a pipeline engineer and a nurse, Lok moved to the Tulsa from Malaysia when he was 5 years old. He credits his father’s prescient interest in computers – and an Atari game – with his career path. Lok’s company, Shadow Health, has plans to develop applications for other occupations that rely on face-to-face communication. *This episode was originally released on January 2, 2019.*
TRANSCRIPT:
Intro: 0:21
Inventors and their inventions. Welcome to Radio Cade, a podcast from the Cade Museum for Creativity and Invention in Gainesville, Florida. The museum is named after James Robert Cade who invented Gatorade in 1965. My name is Richard Miles. We’ll introduce you to inventors and the things that motivate them. We’ll learn about their personal stories, how their inventions work, and how their ideas get from the laboratory to the marketplace.
Richard Miles: 0:38
Is treating patients like playing a video game and do virtual patients get us better nurses. Here to answer this question is Dr. Ben Lok, Professor in Computer and Information Sciences at the University of Florida. Welcome, Ben.
Dr. Ben Lok: 0:49
Hello. Thanks for having me.
Richard Miles: 0:50
So Ben, we’re going to talk about you in a minute. But first let’s start by explaining what your technology or your invention is, what it does and what problem do you think it’s solving?
Dr. Ben Lok: 1:01
All right. When we think about how do we train people to talk to other people. And we think about that. We’ve been working on that for a very long time and we still have the same technologies, the same approaches. We tend to do roleplaying with other people. And that’s the way we’ve been doing it for hundreds of years. But now we’ve just entered this place where these computer technologies, these simulations where you can see these virtual people much like you see in a movie or a video game. Could we use them to help you practice talking to another person? It’s kind of initially an interesting question. So it’s not too different than let’s say a flight simulator. But if you could practice your communication skills with these virtual people, would that translate to you talking to real people? And so that was a question we started thinking about back in 2004. So we’ve been thinking about this for a long time. We wanted to study the science behind it and really evaluate whether this was possible. And after about six, seven years, we worked with medical faculty. We worked with educators, we did a lot of user studies, which means we tested the technology out and really be able to understand that people, yeah, did learn how to communicate better using virtual people. Then we decided to form a company named Shadow Health and we said, can we bring this out to the masses? Can we get this rolled out into a lot of universities so we can be helping the new beginning nurse. Think of yourself as a nursing student just starting nursing school and you have to and learn how to talk to people. And so we said, hey, can simulation play a role here? That’s what Shadow Health has now rolled out to lots of universities and colleges across the country. But the goal is again, talk to these virtual people and it’ll help you become a better communicator.
Richard Miles: 2:28
So just so our listeners understand what exactly we’re talking about, they are siting in front of a computer screen. It can be in a classroom, it could be at home on their laptop, whatever. And in front of them, I think your first patient with Tina. And so they see a patient on there. And what happens after that? They look at the patient and then who starts the conversation and how?
Dr. Ben Lok: 2:47
Right. So think of yourself as a nursing student. You learn a topic in class and then the teacher says, okay, now you need to practice talking to a person about it, right? So let’s say you learn about cardiology or respiratory or something like that. Then go home, open up your laptop and log in and you will see a virtual character on the screen. We have many different virtual patients, different genders, ethnicities, backgrounds, ages. And then you can either type or talk to them and they will respond to you. And so the goal is to ask the right questions, observe how she or he is responding to you gesturing and whether they’ve got facial expressions and things like that. Ask questions, empathize, educate. When necessary. But really practice communication skills because we know that at the heart of good healthcare is a relationship that a patient has with the providers, right? And so if you can build empathy and rapport with a patient, we know that you get better health outcomes. Patients get better because they reveal more, they feel secure and they provide you with more information. Getting that’s hard. Practicing that as hard. So think about it. Beginning nurse, you have to know so many things. How can we use technologies now to help you prepare for that? And that’s what this technology is, so again yeah, you open up your laptop, you see a character, you type, you talk to them. But the important part is after you’re done, you get feedback on how you did. Did you address her concerns when she raised that that hey, I didn’t understand the words that you said or my mom got sick from this, I’m scared I could get sick. So these are challenges that are really hard to deal with whenever you do it face to face. But if you could practice it in a place that’s controlled where you can control the pace and really get feedback on how you did, we think that that will improve your communication skills.
Richard Miles: 4:19
And what really impressed me the first time I saw the technology a few years ago was the fact that you design these characters so that they’re supposed to be like an actual person. So if you slip into medical jargon, for instance, they won’t necessarily understand what you’re saying as you said, or they’ll give you half answers, just like a real person might not fully explain their symptoms and you have to ask followup. And that’s deliberate right? To really challenge the student, to frame their questions and their followup questions in a way to get the information they need.
Dr. Ben Lok: 4:49
So when you see a nurse or a nurse sees a patient, that patient has a whole history of what got them to that point, right? They’ve got a belief system, they’ve got a family, they’ve got experiences before that they’ve run into doctors and nurses in a certain way. And all our nurses and doctors, they’ve got a practice communicating with that in mind. So each of us has our own backstory. That’s the challenging part, right? So even us coming to this conversation right here, we just happened to meet at this one point right now, whenever you’re asking me questions for like right now you’re interviewing me, you’re thinking about…
Richard Miles: 5:21
And it’s a virtual reaction, I don’t really exist.
Dr. Ben Lok: 5:24
You are very realistic. You know, the thinking is that you need to be able to unpack and understand how this person got there. If this person is scared or concerned, there’s a reason why. And that’s what a great nurse can do, right? They ask the right questions and to be empathetic at the right moment. So if you say you’re afraid of a procedure or you hear a term that you’re not familiar with and you get nervous, the nurse has a really great opportunity to step in there and comfort you and say the right things and to help explain things. But that requires understanding all this backstory. And so we’ve intentionally built our characters with a store in the back that helps frame who they are today. So if you asked somebody, for example, do you have allergies? If they’re not medically verse, they might say, I’m allergic to cats. Well that’s part of the story, but you can be allergic to medications and latex and so many other parts and so we wanted to build that into our products so that people do get practice understanding the whole story that we’re each individual people I could be male, 43 Asian, but I have a whole backstory. I moved to the US when I was five, for example, so I grew up in Tulsa, Oklahoma. Happened to like a little bit of country music, which you might not guess when you first see me, but we all have that story. My mom’s a nurse for example, and that’s why I’m very much into the healthcare profession and trying to help them because I’ve seen my mom do that as a nurse and so these are all part of who I am and how I got to this point and that does build that story in for good healthcare to happen. Nurses have to quickly get at that and try to be able to understand that we’re each individual people.
Richard Miles: 6:47
Right. We’re actually going to talk more about your personal story in just a minute. But one follow up question about the technology itself that I think is important for listeners to understand is this is obviously not alive in the sense that your virtual patient isn’t being controlled by somebody who’s answering. So when you first started out with Tina, how many questions could she answer, you know, was it 500, was it 1000 questions? And then how did you make Tina smarter?
Dr. Ben Lok: 7:11
That’s a huge problem in computer science. It’s a field called artificial intelligence or some sub areas, natural language processing or understanding. And so in that space, when we first started creating Tina, my area of specialty is actually called virtual reality or human computer interaction. And I really wanted to study how people react to these characters. So at one of the nice things about these virtual characters is they can’t talk about everything unlike Siri or Alexa that you have at home, which see me, they can talk about a variety of topics. These virtual patients only need to talk about why they’re there in the clinic, right? So if you asked Tina Jones, which is our first virtual patient, what do you think about the gators this year? What are their chances? She’s not going to have an answer.
Richard Miles: 7:50
She’ll stare at you blankly. Which is a normal reaction, right?
Dr. Ben Lok: 7:53
As you know, that’s not really something, I’m here to talk about it. So it really kind of gets you back into this scenario. So we had a very constrained part and the nice thing about a good interview with let’s say a nurse in a patient, is it that it’s slightly structured. You might not know that as a patient. Sure. But there is a structure behind it, right? They’re progressing through stages. And so we leverage that in the technology because we can anticipate what you’re probably going to say, right? You’re not talking about anything under the sun. You’ve got a few things you can talk about. Allergies, medication, family history, they’re constrained, right? So we build these areas that she can talk about. And when we were first in the research lab, she can answer a few hundred questions and she could talk for about eight, 10 minutes. But now with a commercial system that is flushed out and we’ve got professional writers and folks dedicated to building out our characters, most of our characters can answer hundreds of thousands of questions they learn from everybody that they talked to. And there’s some pretty sophisticated backend stuff that really tries to get our characters to learn from everybody that goes through. So as people more and more people talk to our characters, they get progressively smarter. They’re able to expand what set of questions they can respond.
Richard Miles: 8:57
So according to your patients, how are the gators going to do this season?
Dr. Ben Lok: 8:59
Um, I am, I’m very optimistic. I’m very, very optimistic. I hope it will be good year.
Richard Miles: 9:08
Then you’d mentioned earlier that you grew up in a suburb of Tulsa, Oklahoma, but you were born in Kuala Lumpur in Malaysia. How did your family come in the United States? Tell us a little bit about growing up in Tulsa and what was that like?
Dr. Ben Lok: 9:19
So my dad was a pipeline engineer, so he was an electrical engineer by training, and he built pipelines. And so born in Malaysia. That’s where a lot of my extended family still live in and my Dad was working for an oil and gas company out in Jakarta, Indonesia. He had an opportunity to move to Houston or Tulsa. He knew he wanted to somehow make it to the US because there’s opportunities there for his family. And so he chose Tulsa. I’m not sure why, but they love it. They love it. They love Tulsa. They’re still there. We moved there in 1981 and my parents still live there 37 years later. They’ve got their group of friends. They absolutely love it there. My Dad an engineer and so it was just company, his oil and gas company because Tulsa was the oil capital of the world in 1981 and 1982 there was an oil crash. It was a very scary time for my dad because he was an immigrant with two young kids at the time. I think my sister was on the way and I was five from the move, so my dad, I always tell a story. He saw that the computer was suddenly became something that you could buy for the home. It’s extremely expensive. At the time, I think it was like $3,000. Back in, 1981 that’s an enormous amount of money. It still is a lot of money today, but it was as I recall, several months salary. But he said, you know what, there’s something going on here that if you could learn this, you would have a significant advantage. He saw like the draftsman, these jobs are changing. So he invested money and bought an Atari 800 way back when and he said, “you know Ben”, I was the oldest of his kids. I was five at the time he said “if you learned how to use this. I think that this will serve you well.” He didn’t know why right back down. He just, he was electrical engineering by training, so he liked the electronics. But I learned how to start programming when I was five and he really sat me down and we learned together. It was something that you know when I look back on it. It was just fantastic. But the reason I tell that story was two years later was when the oil crash happened. So my dad had just moved his whole family here. Didn’t have much money. He was spending most of his money back home to his parents to pay for his brothers and sisters go to college. And we had just moved to this country and he just spent all this money on a computer. But when the oil crash came and large swaths of his company were being laid off, I think like 60 70% of his company was being laid off and he’d be an easy person to let go, that he knew computers actually saved his job and allowed us to stay in the US and…
Richard Miles: 11:23
Do you still have that Atari?
Dr. Ben Lok: 11:24
My dad probably could be on a show of hoarders and so it is probably somewhere, but I always give him credit for taking a big chance out there. And so yeah…
Richard Miles: 11:34
That was not entirely obvious.
Dr. Ben Lok: 11:36
It was not, it was not. Anyways, because he saw a technology coming down and he invested the money and time into learning it. He altered the course of our family’s history, but it also gave me a passion to say “Hey, this tech can help people,” right? So this tech, it’s cool for games. I love playing games. I love using…
Richard Miles: 11:53
Did you do a lot of gaming when you’re a kid or…
Dr. Ben Lok: 11:54
Yes, I still do, I’ve got my own kids. We play Mario Kart together and it’s great, but it has an entertainment role. But I also think that it’s got an extremely untapped education role. When I say educational, this means just putting a computer in front of a lot of people. That doesn’t mean education, right? So it’s to say, “Hey, we’ve got this new platform. It can deliver experiences in a new way.”
Richard Miles: 12:14
Right.
Dr. Ben Lok: 12:14
“What can we do with it?” And I wanted to innovate in that space. And so that’s really what has been my passion as I went through school, through high school and growing up in Tulsa. And then I went to the University of Tulsa as an Undergrad. I said, I want to go college as far away as possible. And then…
Richard Miles: 12:29
And then 10 blocks, right?
Dr. Ben Lok: 12:31
That was actually a few miles down the road from my parents. A lot of it was the fortune of scholarships and reality finances. Right. But then I went to the University of North Carolina for graduate school then.
Richard Miles: 12:40
And what was your undergraduate major in?
Dr. Ben Lok: 12:42
Computer Science. Computer Science as an Undergrad four years there and then went to the University of North Carolina Chapel Hill so I’m a big Tar Heel fan. Yeah, I think they’re going to do quite well this year. I think I’m going to program that into our carriers.
Richard Miles: 12:53
Well what do your patients say, thats important right?
Dr. Ben Lok: 12:53
They’re fantastic. Right? Go Tar Heels. Uh, and I spent… Yeah so I did two years for a masters, then another three more for a Phd. And we were all done in computer science with the, “How can we use this?” There’s a branch of computer science thats very theoretical. There’s also an applied side. And that was the area that I wanted to focus on was the applied, where can we apply these computing to solve some of the problems that five, ten years ago we didn’t have the technology for. So in potentially now that we have this, what new doors could now be open. We’ve got these new tools that have just entered our tool belt. So that’s been a very fun place to be in because the pace of advancement has been so fast, right? Just a few years ago, we didn’t have things like in our homes that we could talk to. That was not a thing even a few years ago. But now we find that by having that we can, you know, we open a lot of doors, but also some really interesting side doors. So for example, we know that things like Google home or Alexa, they can help people on the autism spectrum, for example, learn how to communicate. My son’s on the spectrum and he learns so much about how to communicate, how to phrase questions and things like that, using this new technology. So that’s an example. I’m always fascinated when they created Alexa, that was not their main use case, right? But we’re finding out how all these technologies can have these residual sides. So studying, that’s very exciting. But also innovating in that because we can use it for good or also use it for not so good. So I want to be on the side to help study it so that we can make sure we’re making good decisions with it.
Richard Miles: 14:10
So Ben, that’s a perfect segue because I was just about to ask you about Shadow Health.
Dr. Ben Lok: 14:14
Uh Huh.
Richard Miles: 14:14
And you as a company to deliberately picked nursing as your market that you’re going to go after applying this technology. But as you just mentioned, there are some other obvious, obvious to me, but you tell me if they’re obvious, applications in which you had the same dynamic of communication skills is important and you want to give people an opportunity to practice that. So are there other markets potentially that Shadow Health itself or companies like Shadow Health could get into using the same core insight? Here’s how to practice a communication. It’s sort of a set format and I’m thinking of like even paramedics or firefighters or anybody who has to communicate information to another person, and it’s really important that they get it right. Tell me about that.
Dr. Ben Lok: 14:55
Right. Go back to the very beginning when I said the, the core thing we’re thinking about is how do people who need to communicate with somebody else, which is most people. Most people have a job where they have to communicate with somebody as part of their job. How do they get better at it? Right? That core question is in a lot of places where it’s somebody working in retail and somebody who’s working in banking or finance or an education, the military, there are a lot of folks who have communication needs and so if we could practice that, that would provide significant value. It plays a role in one’s education, right? So absolutely. We, we started in nursing. The main reason we started in nursing, one is that there are a lot of nursing students and so it was just an addressable market that could sustain a company getting started. But we do see the technology having two places, one in healthcare, we want to definitely be able to work with other domains within that space. And we’ve had people in like physician assistant or some medical schools that have said, hey, this technology also applies there, so we do want to see more coverage within the medical space, but then there are other places too such as the education market, right. Thinking about where we could practice communication skills, even K-12 for example, as a place to say, wow, there’s a lot of places one needs to practice. The simple act of, but not so simple act of talking to somebody else. Not only exchanging information but be able to understand cues and be able to understand how to have a conversation to get important information across.
Richard Miles: 16:13
I remember when I first learned about Shadow Health, I was fascinated by the business rationale for the model in that in nursing education you had this bottleneck around the ability to conduct patient interviews and as you said earlier, the traditional solutions were either roleplaying, which was expensive because you’ve got to hire actors to play patients and so on and obviously can only do that during business hours and so on or on the job…
Dr. Ben Lok: 16:38
Shadow. That’s the shadow of shadow right there.
Richard Miles: 16:40
Which wasn’t ideal as well because you’re trying to in a real life environment, teach somebody these important communication skills. So that the ability of Shadow Health, where you came in from a business perspective was you’re providing a solution to solve that bottleneck problem. Now a nursing student at 11:30 at night on their couch with a laptop can practice and doesn’t have to rely on actors and so on. So have you identified other markets opportunities like that in which there is a specific hurdle in the training that their means are either too expensive or not very efficient? Is there anything like the nursing market out there that you’re thinking of getting into?
Dr. Ben Lok: 17:17
I want to highlight one of the big bottlenecks is actually the educator, right? If you’ve got a classroom of 120 students and you’re one teacher, how do you provide feedback? How do you provide the individualized attention and experience? Because that’s where a good learning happens, right? Whenever a teacher is able to focus on you… So if you’re gone to a, let’s say a swim clinic. It’s great when the teacher shows everybody in the class how to swim. But if the teacher can critique you individually and tell you what you’re doing, it changes, right? Golf clinics or running clinics or whatever. So the teacher’s really a big bottleneck, right? And actually where I’d like for people to focus on, so empowering the teacher to be able to give out individualized feedback to large numbers of students and the number of students are only growing, right? We got more and more nursing students coming in, but the challenging bottlenecks, the educator being able to provide that feedback and where this technology can really come in, it’s not only do you get to talk to a virtual character, that part’s great.
Richard Miles: 18:08
It captures the conversation.
Dr. Ben Lok: 18:11
It captures, and then after…
Richard Miles: 18:12
It analyzes the conversation.
Dr. Ben Lok: 18:12
Right. That point afterwards, it’s actually where the learning really happens, right? It’s getting feedback on how you did opportunities you missed, here’s what you did, here’s how an expert would have done that, right? So here’s an example. If the character expresses concern around something, I don’t know where that can afford that. Okay, how do you respond? That’s great that you had to experience, but afterwards when you look, go back to that moment and you can see not only how you did but how an expert, “Wow, that was really hard to answer. I don’t even know how I would answer. Oh, that’s how an expert would do it.” That moment is really, really valuable and that’s the part we’re trying to get to more students is to get the individualized feedback to people because we know that we can give that to you right after, not when your educator unfortunately, it’s super busy. As a professor myself, I would love to be able to sit next to all my students and provide that. It’s just not feasible, but technology could be an extension of the professor to give that feedback at that moment. That’s critical and then with that information now you can come talk to me as a professor and then we can have a conversation. It’s a lot deeper than a very superficial level because we have a very careful record of what you’ve done and so it’s really the bottleneck of David Messiah is the CEO of the company likes to say “We’re bringing that relationship between the teacher and the student back to a one on one.”
Richard Miles: 19:21
Does the software have tools so that, let’s say I’m your student, right, and I’ve interviewed a virtual patient and the patient told me they had swelling in their ankles and I miss that or ignored it or whatnot. Does the software flag the fact that I got it wrong and then you as the professor would get a report saying, hey Richard, it’s screwed up here because he should have followed up on that, or would you have to go through my entire transcript and find out all the bone headed things I’ve said or didn’t say?
Dr. Ben Lok: 19:48
That’s a great question. So when you go through and talk to a virtual patient, both in our software and actually other companies in this space, because there are a lot of other folks who have seen that this is a great market to be in. Yes. You get feedback afterwards and say, “Hey, these are opportunities. You missed these” or “Good job. You got these parts, right. You missed these opportunities. Here are other ways that other people could address that.” The educator not only sees that, but they also see that for the entire class. Right? So if you have hundreds of students, you’ve got the next class, what are you going to talk about when you get up in front of them? If you could see that, “Hey, did you know that 80% of your class forgot to ask about allergies?” Let’s say drug allergies or family history questions. Because you’ve been so focused on this. How do you even know that you’ve got a class that might be focusing or forgetting to focus on a topic so the software not only captures and gives you individualized feedback, but also gives class feedback to the professor so that professor can look at it and go, “I thought I covered this thought I said it a hundred times but I guess I’ve got to say it one more time.” to cover topics that the class as a whole is or is not proficient at a level that you would want them to be in. So it provides both individual but also class size feedback to the educators.
Richard Miles: 20:52
One final question on Shadow Health. For the longest time Shadow Health was sort of like the hottest startup in Gainesville and I’m not even sure it’s a start up anymore. But you’re in a lot of universities to tell us what the business environment looks like with Shadow Health.
Dr. Ben Lok: 21:05
Yeah. A lot of folks ask how did the Aha moment happen for Shadow? And the exciting part is that there was no Aha moment. It was a flywheel that kept speeding up. Right? If you read the book Good to Great, for example, the top out of fly wheel that picks up speed. So from 2004 through 2010 we just did research. I had no thought about starting a company. I wanted to do good science. I wanted to do to look at this new invention and really be able to study and we were running it at the medical school, at the University of Florida and then the Medical College of Georgia, we built a consortium of folks there. Then in 2010 I was approached by entrepreneur in town, David who I mentioned before and he was looking at technologies and we had a meeting and he did market research and saw that there was an addressable market with the nursing folks. He said there was enough folks and not only that, if you can improve nursing in this country, that’s actually huge. It potentially is bigger than improving elms, any other profession because you can affect healthcare. Each nurse, there are estimates that they see about 2000 patients a year over a hundred thousand nurses, user software. New students use it every year.
Richard Miles: 22:01
And they’re one of the primary sources of information for the doctors, right?
Dr. Ben Lok: 22:04
Absolutely.
Richard Miles: 22:04
Doctor’s rely on them very heavily.
Dr. Ben Lok: 22:06
You’re talking about hundreds and millions of patient encounters. So I’m very excited that potentially most of us have interacted with a nurse that had Shadow Health training. And hopefully that because of that, when they talk to you, they think back about asking that extra question or observing some response that you gave and connecting it to the feedback that they got. Right? And they ask you an additional question or you remembered to process something a certain way. So we started 2011 and first semester that we sold was 2012 we had six universities that said, yes, we’ll buy it. Even though we didn’t have the software fully done, which showed us that there was really this hunger for this technology. But now we’re in over 1500 universities and colleges and as an inventor, the opportunity free technologies that you know, has a good science foundation to be able to be out there and effecting people. Again, we’ll have over a hundred thousand folks using our software this year. If they each see 2000 patients, that’s 200 million patient encounters that you think could be slightly improved. Right? Not every single one would necessarily be better, but if you could help 1% of those, that’s a significant improvement and potentially affecting healthcare in this country in a very organic education driven way that I’m very excited about. It’s driven by new way to think about technology. These video games, I love video games, I love playing them. These characters that you see there, instead of shooting them with shotguns and running around with them, why don’t we talk to them? That was the genesis of all of this and so I try to explain to my wife when I play video games, it’s just research for… it’s just future research. But it’s surprising how people react to these characters when they see these characters. They have a reaction when a nurse sees a virtual patient, they want to apply their nursing skills to these characters and as a scientist, I always find that very fascinating and like you said, there’s so many more markets and areas that could use this practice. Communication skills and we’re learning a lot from the nursing market so we can apply that to other places.
Richard Miles: 23:48
So we’ll know Shadow Health is truly successful when they do a poll and they find out that a disproportionate number of nurses are Florida Gator fans, sneaking in the subliminal messages.
Dr. Ben Lok: 23:59
All right!
Richard Miles: 23:59
Ben, thanks very much for joining me. Fascinating discussion and hope to have you back on the show at some point.
Dr. Ben Lok: 24:03
All right. Thank you so much.
Outro: 24:06
Radio Cade would like to thank the following people for their help and support. Liz Gist of the Cade Museum for coordinating and inventor interviews. Bob McPeak of Heartwood Soundstage in downtown Gainesville, Florida for recording, editing and production of the podcasts and music theme. Tracy Columns for the composition and performance of the Radio Cade theme song featuring violinist Jacob Lawson. And special thanks to the Cade Museum for Creativity and Invention located in Gainesville, Florida.
