Busting Real-World Data and AI Myths

Hi, I’m Daniel Chancellor and welcome to Real-World Data After Dark. We are here live in New York City and we have not just one, but two senior leaders from Norstella who are experts in this space who can really share some of their insights into everything that we’re seeing in that real world data space. So my name is Daniel Chancellor. I’m VP of Thought Leadership at Norstella. To my right, I’m joined by Kris Kaneta, who is our chief product and innovation officer. Welcome, Kris.

Thanks, Dan.

Thrilled to be here. And Ted Search to my left, who is the Chief Real-World Data and Analytics Officer at Norstella. Welcome, Ted.

Thank you very much, Dan. Excited for today.

AI and real world data just go together hand in hand to borrow an American analogy as we’re here in New York, like peanut butter and jelly. With all of the progress that we’ve seen in the frontier models, and I know all of the cool stuff that Kris and his team are building, it’s just the natural conversation to have. So yeah, let’s start with you, Kris. What does it really look like when AI gets layered onto real world data?

So first, I think it’s important we recognize it’s not just AI being layered on top, but it’s about really being thoughtful as to how that real world data is structured and linked and underpins any endeavors in AI. So really to me starts with, do I have a really good understanding foundationally of what’s going to sit underneath any AI pursuits that I might be investing in? And then from there, as we think about AI sitting on top, I mean the applications are endless and it could be as foundational as thinking about understanding patient populations and understanding the dynamics of specific cohorts. It could be thinking about how do I understand and sub-segment patient populations and really get close to understanding clinically as to what’s happening around those patients? And then it also comes to thinking about how am I looking at the market and my go – to-market motions and my promotional commercial activities and understanding what are the next best actions that I can trigger as I think about reaching the right healthcare professionals and providers that are treating the patients that are most appropriate for whatever it is that I’m developing and investing in.

And then ultimately I think about how AI is going to create that first set of signals. But ultimately it comes down to really strong clinical and commercial judgment as to the folks that are really immersing themselves into these AI models to really take it to the next step and understand, okay, these are the real implications in practical terms, in organizational terms, in regulatory terms. All of these things need to kind of come together with, yes, AI surfacing and triggering insights and next best actions, but real clinical context and expertise to help make that reality.

Dan, if you don’t mind, I’m going to agree with my colleague and friend Kris here that it’s a fact and I’ll take a little additional perspective on it. Think about unstructured data that we have in EMR. For a decade now we’ve wanted to see the deep rich insights in there, but it would take months upon months, a hundred PhDs, physicians reviewing those notes to be able to get really the valuable content out there, biomarkers, tumor stage, physician sentiment. AI has just unlocked that world. So when you look at patterns you can generate, insights that you can pull out now in a matter of time, now with the right prompts, still having those PhDs as physicians create the right prompts that could prompt these models to extract out that tumor, that disease state, the therapeutic areas, the rich content you can’t get from structured data now in a format that can be impactful for clinical decisions, commercial decisions, impactful for a patient getting on the right therapy at the right time, absolutely a fact what AI is doing to transform that and really benefit patients’ lives.

All right, thank you so much. Well, I love the introductions. We are often rolling. And you’ve kind of teased some of where I wanted our conversation to be heading. Before we get too far ahead of ourselves and too excited about what we can deliver with our real world data and our artificial intelligence, I wanted to make sure that we are keeping our fee on the ground first and you’ve been bringing in facts. I’ve got a few questions I wanted to ask both of you and have a little game here. We know fact or fiction. And Ted, I’ll stay with you because you’re talking about this already. Is it fact or fiction that AI can identify patterns and real world data quicker than traditional methods?

Absolutely a fact. As I talked about before, if I take a specific answer, example that I provided before on unstructured data, the fact that we can now leverage AI models with the right clinical rigor behind them in order to extract out information like biomarkers, tumor stage, why a physician is choosing drug A versus B, we could never unlock that before. That allows a patient that we all serve to get on the right therapy at the right time, have access to the right clinical trial at the right time, and provide the partners that we work with, with insights and really analytics that they can use to generate evidence they never had before. So absolutely a fact and that’s one specific reason why that we’re seeing practically.

So you might see others making claims about three months or three days or something, but you’re saying with our information, it’s not even something that used to take three months before, it just wasn’t possible before because of the sheer impracticalities of.

Yeah, I think if you really just take Norstella as an example, Ted, as we look at every single record, every test, every treatment, every episodic component of a patient journey, you add all that up, that’s literally trillions of records. Or you take Ted’s example of unstructured notes, I’m now thinking about billions, literally billions of unique notes. And so there is no pre-AI world where that becomes something I can sift through in a matter of days or even potentially weeks. Now, certainly machine learning has advanced over the last few years to get us closer, but where we are now to be able to really drive and create a context layer around a specific disease or on a specific patient population, it is unlike anything we’ve ever seen before. And I think even more important is just wait six more months. The implications are going to be fast and furious.

But at the same time, I also want to call out that this is all possible because we have some of the brightest clinical minds, brightest data scientists who are not only technically brilliant, but also informed and capable as it relates to the context of life sciences, as it relates to the context of disease progression, as it relates to the drug development pipeline clinical trials to bringing a drug to market. So all that data and that context layer doesn’t come together without really bright people who understand how those pieces fit together.

Yeah. So fact or fiction then, AI produces this work autonomously then without human involvements.

Fact AI can take a very compelling first cut, but I think we all know, I mean, even the simplest queries sound credible, sound convincing. In fact, I have a set of agents that run every morning that kind of inform me as to what’s happening in the market and what’s happening in my calendar, what’s happening in the landscape. And even just this morning, I noticed a few things that sounded really credible, but the agent was filling in gaps that wasn’t out there in the real data to make the story flow. And I called it out and said, “Oh, you’re right. I totally made that up.” And so you’ve got to have that human in the loop. You’ve got to have that credible clinical expert, that data expert, that workflow expert that says, “Is this actually real? It sounds credible. It sounds compelling, but is it real?” So fact, yes, on a first pass and identifying those initial signals, fiction that I would just trust it completely autonomously end to end.

Yeah, I’m completely aligned with what Kris said. I was almost going to say fiction to just contradict him, but I agree with all of the facts that he provided. If I look at it at face value, the only reason I would say fiction is because one, as Kris was talking about, the old days of bringing data in, having your technical team then having to surface and align this data, have a data scientist pull it out. So then you have a query that you have medical specialists, analysts generating evidence out of there is really gone with AI. What it does is it surfaces those initial insights, those cohorts of patients that you could really analyze. And what it brings to light very quickly is the experts then that can take those insights and generate them into actual evidence. And that takes specialists to do that. You’re always going to need that specialist in the loop to be able to conduct and get that evidence.

So reason I say fiction is Kris is right. It is a fact completely that it’s going to generate insights quicker than we can before fiction because it’s always going to involve a specialist that then now can ask 10, 20 questions at the same time on the analytics coming out and generate very significant evidence as a result.

One final one before we move on, perhaps a quick fire. Fact or fiction AI can accelerate clinical trials. What’s your takes on that one?

Fact and absolutely happening already. I don’t know. Kris, if you want to add to that, I would say fact. Yeah,

I mean, absolutely. Absolutely fact. I mean, from some of the work we’re already doing in advising and prescribing an investigator selection to protocol design all the way down to from a real-world data perspective, identifying the right potential patient groupings for patient engagement and recruitment. All these things are already real and happening out in the marketplace. So 100% can absolutely accelerate clinical trials.

And if you think about even clinical trials, patient recruitment, or thinking about feasibility of a clinical trial, to Chris’s point, a lot of the work pharma’s doing is in oncology, in a rare disease, specific targeted medications for very specific patients. In the past, without AI, it was very hard to surface, again, that biomarker data, that tumor stage, which is so important to understand what’s the right patient for this clinical trial, this ultimate medication. The fact that we can do that at scale with AI and then really understand the patient population in a timeframe that we can act is already helping with looking at what is the right medication for the clinical trial, what is the right patient population? And then ultimately working with investigators, physicians to help drive the recruitment for clinical trials, which is so important.

Yeah. And I’ll add one more thing just to kind of make this real. It’s a fairly well understood data point that as many as 35 to 40% of clinical trial sites never actually recruit a patient. And what our research has found is that humans by and large are actually terrible predictors of key measures when it comes to clinical trials, such as will this investigator actually recruit a patient? Well, we know 35 to 40% of them actually never do in terms of predicting the duration of a clinical trial. In fact, our work has found that when we feed all that data into our models, we’re actually able to predict the duration of a trial 40, 50% better than what would be your historical predictions. Obviously really intelligent human experts, but there are just so many variables to consider. What’s happening in the trial landscape around me?

Is there clinical trials that are competing for the same pool of patient population? Does this investigator actually treat patients that are going to meet the specific criteria that I have in my protocol? So all of these things are massively complex data points unto themselves, but leveraging AI to actually bring this all together is something we’re already seeing have massive impact.

And it’s such a great point Kris makes just that. And why does that matter? Because for patients, it gives them another care option that they necessarily didn’t have before. If you think about it, prescribers, physicians that see these patients, they many times don’t know where clinical trials are taking place, what trials are taking place. Are they in a close proximity to where I could drive my patient to go? When I see my patient, do I understand what’s available to give them this other care option, especially in areas like oncology, rare disease? So the impact AI has, yes, is on the data. Yes is for driving clinical trials for pharma, but also it’s having direct impact for the patients that just have another care option as a result.

Well, thank you very much. I’ve got quite a few more questions. I’m going to scrub them off my page because I think we’ve covered some really fascinating topics. I thought what I attempted or we attempted to bust some myths. We did to a point. Yeah, we did. Yeah, we did. And we tried to keep our feet on the ground, but I think inevitably, I mean your excitement, it comes through, right? It’s infectious. You can clearly tell we’re excited about where we’re going. And with that, the next episode, we’re going to be talking a little bit more about where AI and real world data really are delivering on that hype. So please do join us for that. But for now, thank you Ted and thank you Kris. Thanks, Dan. Can’t wait.