Daniel Nadler on Building an AI-Powered Search Engine for Medicine

Daniel Nadler: Where I want to add value to society is I want to create the MRI of medical knowledge that allows a physician to scan the full body of medical knowledge at the point of care. 

[MUSIC INTRO] 

Meena Flynn: Welcome to Goldman Sachs and Talks at GS. I'm so pleased today to be joined by Dr. Daniel Nadler, co-founder and CEO of OpenEvidence. Daniel, thank you so much for joining us today and coming back to Goldman Sachs. 

Daniel Nadler: Well, thank you for having me. I never really had a real job other than companies that I've founded. But the closest thing that I ever had to a real job was when I started my first company and Goldman basically incubated that company. It was called Kensho Technologies. So, this is kind of where I grew up professionally. And so, it really feels like a homecoming for me. 

Meena Flynn: Thank you, Daniel. Let's start from the beginning. You know, you sold Kensho in 2018 for at that time the highest amount that any company had gone for, which was $700 million. You're in your 30s. You're paddle boarding, you're studying the classics. I guess then, like fast forward, it's like you're going through COVID, et cetera. How do you actually decide to start OpenEvidence? 

Daniel Nadler: I spent the time in between companies continuing my humanistic education. I went back to reading the classics again. Studying Ancient Greek again. I actually decided I was going to learn to draw well. So, I studied drawing with a teacher from the Royal Academy in Britain going through like the basic motions of how do you draw, shade on an apple and these sorts of things. And it's a scary thing to accept that you can actually learn anything that you want to learn because that like feels like a lot of work. 

But if you accept that deep truth that you can learn rocket science, that you can learn medicine, that you can learn all of these things, then it's extremely empowering. 

So, I was doing a lot of that between companies. COVID hits. And it's very clear to me that my second company will be in medicine. And I saw the horrible images that were coming out of New York City. And I just felt an incredible sense of activation, motivation, and purpose. 

And there was one particular fact that I came across. Because at that time, the sort of minutia of medical literature and academic publishing became front page news in The New York Times and The Wall Street Journal because of COVID. And they published a fact that the COVID literature had gone from, you know, obviously no papers, no one knew anything about COVID, to something like 30,000 papers in a matter of months. And it went from, in other words, there's no needle, to now there is a needle, but it's in a haystack that's doubling in size, you know, every like 30 days. 

And that felt to me like as a framework for a problem, a lot like what I had spent time thinking about in my first company. Because honestly, you know, information science is information science. Information overload is information overload. I don't think any Goldman Sachs, no matter how good you are, any Goldman Sachs analyst believes that you can cover, you know, 100 companies extremely well and in detail. And yet the more I sort of pulled the thread of this, that's exactly what we expected physicians to do. And it wasn't just with COVID. 

There was one cancer drug in 1950, which wasn't really a cancer drug, it was nitrogen mustard. It's basically a chemical weapon. And they saw through the experience of battlefield doctors treating American soldiers who had been gassed in the First World War, and some in the Second World War in the Pacific Theater, that their white cell blood counts dropped precipitously. 

And some very smart scientists theorized that that might actually be useful if someone has leukemia, because these are mutations that occur in the white blood cells. Which turned out to be true and effective. Today, we're living in the golden age of biotechnology. There are 100 cancer drugs. So, your coverage universe is now 100 drugs that you need to know at least as much about as you know about a company, okay? They need to know every side effect, every drug-drug interaction, where to sequence it in a cocktail, you know, does it come first line, second line? 

There's an entire universe of details they need to know about these drugs. And their coverage universe is 100 companies or 100 drugs today. And an oncologist, at least what we expect of them in society, they can't just break it up like Goldman does and they say, "Okay, you cover NVIDIA, you cover SpaceX, you cover Lilly." They're all expected to know all of this, right? That's an entirely impossible expectation. 

So, that's why OpenEvidence became this enormous hit. Because it was this brain extender. You know, for a finance audience, this is Bloomberg Terminal for physicians. Which allowed them to work with a computer and really rapidly call up all the complexities of every single drug in their coverage universe. And things beyond drugs like diagnosis and so on. 

Meena Flynn: If you had three things that you want your company to do to help solve that problem, and others, what would it be? 

Daniel Nadler: You've all heard as patients, get a second opinion, right? A study of second opinions found that 88% of second opinions changed from the first opinion. One in five were completely overturned. Only 12% of second opinions were the first opinion. 

So, there's enormous unwanted variance just on the basis of chance, rolling the dice, which physician you happen to see, largely when they went to medical school. Because keep in mind, this golden age of biotechnology that we're all so excited about, it means that all these amazing immunotherapies and all these things that we want our doctors to treat us with were FDA approved long after they left medical school. Okay? 

So, what ends up happening is there's this enormous transmission lag between what's called, you know, bench and bedside. In other words, we are usually getting the benefit of frontier medical knowledge from 17 years ago. Imagine an AI if there were a 17-year lag between the frontier model and the application of it showing up on your desktop. That would be absurd, right? We don't accept that in AI and computer science or in work. But in medicine, which is, you know, life and death for many people, that's just the reality. 

So, our missions are to make it so that the vast majority of second opinions are not overturned, which just means that there's no unwanted variance in the system. Everybody is singing from the same sheet where reasonable. There's still going to be 5% - 10% variance because reasonable people could disagree about something. 

And the third corollary, you know, mission is to reduce the lag from bench to bedside from 17 years, really, it should be 17 hours. It should be 17 hours, like just factoring in people need to sleep in, right? It should be that there's a new FDA-approved, amazing drug that will help your mom or your dad or your brother. And the physician, your physician wakes up. And by that next morning, they instantly understand it. That's what it has to be for the whole medical system. And so, that's our mission. That's what we're working on. 

Meena Flynn: And tell us how doctors actually use the tool. 

Daniel Nadler: There are 1 million physicians in the United States. Yesterday, we powered 1.5 million medical license verified queries on OpenEvidence just in the United States. So, this year, there are going to be 300 million Americans who were treated by a physician who used OpenEvidence to make the treatment decision. It's nation-scale use already. It's become the default operating system of medical knowledge in the United States. 

So, the reason I say all that is it's hard to generalize at the scale of hundreds of millions of clinical consultations and just pick, you know, one or two representative ones. What I will say is our usage skews complex specialties. So, we're very over indexed on cancer, on Alzheimer's, Parkinson's, neurodegenerative disease. And it's under indexed on family physicians. You know, "I have a rash. I–" And that's simply because they know how to treat those things. Our users are physicians, right? 

So, they're using this for these very complex, multiple comorbidity treatment decisions involving usually very new drugs. And we're usually the bridge. We're not coming up with the answers ourselves. We're a search engine fundamentally more than we are anything, which is why Google's our largest investor. And we are essentially helping these physicians search the source. 

The source might be The New England Journal of Medicine, of which we're the official AI partner. It might be NCCN, which is the governing medical society in oncology, of which we're the official AI partner. And we're helping them search this source material to get at the answer. 

Meena Flynn: I think there are a lot of investors in this room that would say, "Well, can't I just get like The New England Journal of Medicine through, you know, a different large language model?" What's the difference between your model-- 

Daniel Nadler: Not above board. 

Meena Flynn: Well, I think that's an important part. So, maybe we can talk about that. And then also maybe a little bit of the difference between what your model does versus what, you know, people might do if they're searching on some of the other LLMs. 

Daniel Nadler: So, I have to credit Walter Isaacson who wrote the biography on Steve Jobs for essentially, like, this really key dimension of OpenEvidence. So, if you rewind kind of a little after COVID, when I basically got started. 2022-ish before chat GPT. There were very prominent AI companies being sued for copyright infringement by publishers. 

At the time I was reading the Walter Isaacson biography of Steve. And I happened to be on the chapter where he describes the creation of iTunes, of Apple Music. And I'm sure many of you know the story. But for those of you who don't, at the time, basically, people were just downloading-- they were stealing music, right? They were just downloading music for free. Weren't paying the artists. Weren't paying the record labels. They were just pirating music. 

And Steve believed two things. One is that most people aren't criminals and they want to pay artists and their publishers for music. They don't want to steal. And he felt very strongly about that. And he believed the second thing, which is they care about audio quality. And if you just steal music, you're not even going to get the best recording of the thing. 

And so, he decided Apple is going to call up Universal Music and all the big record labels and offer them to just essentially create a consortium around iTunes where they can digitize their music, because at the time it was CD-ROMs and they were trying to defend the CD-ROM business. And he said, "I'm going to give you a business model where you don't have to be on the wrong side of history with respect to CD-ROMs. You can have digital music, but it doesn't have to be stolen. You can get paid for it." And the rest is kind of history. 

So, I was reading this at the time and I said, "Well, I know that AI companies are asserting right to learn. But what if I just call up The New England Journal of Medicine? And what if I call up the American Medical Association? And what if I offer them a consortium where we become their official AI partner? They become economically involved. They become, in some cases, a shareholder in the business." These are not for profits. Okay? 

So, I said, "What if I do with these nonprofit medical societies what Steve did with the record labels?" And I literally did exactly that. And we became the official AI partner of all the leading gold standard sources of medical knowledge. 

And without getting too technical, AI is garbage in, garbage out, gold in, gold out. So, we just put gold in the system, right? It's pretty simple. 

And so, today, a user, a doctor using this, is going to have access to the full text of the phase three RCT, phase three randomized control trial, published in The New England Journal of Medicine yesterday. And when I say full text, you know, a lot of large language models are trained on the abstract, which are the key findings. There's good stuff in there. But a lot of their questions are answered in the methodology section. 

A question might be, for this given cancer drug, were there any women of color in the control arm or in the test arm? Okay, that's not in the abstract. And so, a big LLM company now has the decision of, like, they just don't answer that. They hallucinate the answer to that, you know? Or they steal it. None of those are really good answers, right? 

So, we went with the sort of Apple Music iTunes approach of an OpenEvidence user today has the full benefit of the full text. We allow them to search on it. We allow them to pull out directly the answer. And it's not just the text, it's the tables. 

So, you know, a finance audience can deeply appreciate, you know, that visuals, charts are very helpful. And to this day, you can't pull up a chart on a generalized large language model that is the exact decay curve of how many additional days of progression-free survival does a cancer patient, you know, benefit from on this drug. We actually give them charts. It's like a Bloomberg Terminal. 

So, if I really want to boil it down to simplicity, in speaking finance language, you can call up the stock chart on OpenEvidence of the cancer drug. You can't call up the stock chart, you know, on the other systems. 

Meena Flynn: What's going on in the clinical trials market? And how are you helping from that vantage point? 

Daniel Nadler: So, if you think about how do we get drugs to patients faster, especially cancer patients where there isn't really a cure for every cancer, otherwise, it wouldn't be one of the two leading causes of death. There are basically, like, two limiting factors. One is drug design. And there are incredible companies working on AI drug design. But the other big limiting factor is you can come up with all sorts of cool new proteins and protein folds. But society spends $50 billion a year running clinical trials. And yet, one in three clinical trials discontinue mainly because they can't find enough patients for the trial. So, think about the cost involved in that. 

Despite that fact, of cancer patients who die and who were eligible before they died for a trial that might have been relevant to them and might have saved their lives, fewer than 10% of those cancer patients ever find that trial. Because not only do they not know, but the doctor doesn't know. 

And I think it's a big part of OpenEvidence's future is we are going to match supply and demand on clinical trials. What that will mean from a human impact perspective is a cancer patient or other patients for whom, you know, their last hope is a clinical trial, are going to know about it because their doctor is going to know about it through our system. From the perspective of the makers of medicines, they're going to actually be able to complete more trials. Which is not only economically good for their business, but gets medicines to society faster. Because this is civilization. This is the human experience. This is life and death. And it's going to be a big part of what we do. 

Meena Flynn: Tell us what's next. So, you went from AI and finance to AI and medicine. Where do you think you want to add more value to society? 

Daniel Nadler: I don't want to start a third company where I want to add value to society is I want to create the MRI of medical knowledge that allows a physician to scan the full body of medical knowledge at the point of care and make a better treatment decision and knock medical error out of the top 10 causes of death in the United States. I want that to be kind of the rest of my career. Because at this point, it's really unclear to me whether this is like, you know, is it just a business? 

A lot of people, when they're very successful in business, they become philanthropists. Why would I quit this, sell it, and then give away money that will be more indirect in its impact? You know? I'm going to make the philanthropy part of my career just this, right? 

Meena Flynn: Yeah. Just outstanding. I want to say thank you for Daniel, of course, for being here. But really, what you're doing for society by and large. So, it's just amazing. Thank you. 

Daniel Nadler: Thank you for having me.