Bruno Quinney 00:02
Bruno, hello, and welcome to the DDW podcast, a podcast covering topics around drug discovery and development, pharma, and biotech. I’m Bruno Quinney. Thanks for joining me for today’s episode. Today, I’m in conversation with Suzanne Caruso, President of Strategic Intelligence and Clinical and Regulatory at North Stella. Norstella supports drug developers who are looking to take their treatments from pipeline to patients, finding solutions to complexities in the drug development timeline. One of these solutions is AI. It’s no understatement to say that AI is dominating all our conversations, and drug development is no different. AI is largely being implemented to augment clinical trials and increase R and D efficiency, but why has it suddenly become so crucial. I spoke to Suzanne about what it means for a company to be AI native. How the combination of AI and real world data is transforming clinical development, and how AI is enabling companies to get treatments to patients faster. This is the DDW podcast. Well, Suzanne, thank you very much for joining us on the DDW podcast. Could you start, please, by introducing yourself and talk to us about what your role at North Stella entails.
Suzanne Caruso 01:44
Nice to see you, and I’m Suzanne Caruso. I’m the General Manager of Clinical, Regulatory, and Strategic Intelligence at North Stella, and what that really means day to day is I lead a business that really provides data and intelligence and insights to pharmaceutical companies, CROs, and financial services companies,
Bruno Quinney 02:02
so from your perspective, what does it actually mean for a company to be AI native? I
Suzanne Caruso 02:07
mean, there’s so many different definitions these days, but when a company really reaches being AI native, that means that a lot of the work that, and a lot of the products that they’re putting out, and a lot of the work that they’re doing are actually being run by agents and supervised by the SMEs who have really become the subject matter experts in whatever kind of experience that you are looking for those agents to do, and that work to do on your behalf, so it really is a change from kind of the legacy way of working to trusting that you know agents are going to be able to do a lot of the work on your behalf to a certain level of quality, and you kind of have to oversee that orchestration.
Bruno Quinney 02:46
Do you ever find there’s almost a shift in terms of responsibility? Because I suppose for a long time people have been used to doing these roles themselves, haven’t they? And suddenly we now have the technology and the capability to get these agents to do those tasks.
Suzanne Caruso 03:00
Absolutely, it’s also super cool and super exciting, because you know it takes a lot of the things that you, as someone who’s worked in the industry for a little bit of time, we’re more administrative tasks kind of off of your plate, and it gives you the opportunity to think about something that I’ll say is almost higher value, and in paying attention to, so you can focus on the right things. I do think there is a bit of change, though, in mindset of classically what people were working on in their day to day and what they’re working on now at AI native companies, just the outcome and the expectation of how quickly you’re able to get things done, and hopefully in my world very much impact clinical trials and clinical development, the goal there is, you know, can we figure out if these drugs work, and we want to do that as quickly as possible. So, yeah, I do think it’s a bit of a mindset shift, and I think the industry is kind of really coming around to it. I think that’s both kind of from the bottoms up and the top down. I think both realise the importance and the opportunity that we have in front of us at this particular point in time.
Bruno Quinney 04:04
I’m glad you mentioned the piece about clinical trials as well, because that’s clearly one of the areas where AI is having a massive impact. So, if we start with the idea of clinical development, how is AI, and also in combination with real-world data, impacting clinical development from the very start of the process,
Suzanne Caruso 04:22
I think in clinical development, one of the first things that I started hearing about in AI, even prior to, like, the Open AI, you know, announcement years ago, I guess three years ago now, was all how AI is helping in preclinical, so it was helping with drug discovery, and that continues, right, drug discovery AI is everywhere. The big transition in my world is very much taking AI into the clinical trial phase one, two, and three universe and saying, how can we now leverage the work of AI and real-world data, which is such a massive amount of data that. Make it something that you can actually garner insights from, and we’re able to do this. I mean, what’s what’s so nuts is the amount of data derived from every single phase of a clinical research study is almost unfathomable and un almost impossible for humans to be able to go through in any reasonable amount of time. AI gives us the opportunity to be able to analyse that and make decisions on where to go strategically, but also during a trial be monitoring in real time and saying, is this an opportunity, do I have to make a change here, is this patient the right patient population. So there are lots of different areas where kind of in clinical development AI has stepped in. I’ll just add one more piece here before kind of turning it back to you. I’ll say the other area that I’m seeing AI being used a lot is in study startup, and I don’t know how familiar you are with study startup, but it has been a bear for the industry for a long time. I mean, we have these ideas of these protocols, and we’re like, okay, we write a protocol, we write a consent form, we get, you know, HCPs that are investigators interested in doing the study, and you should be able to just turn it on and next month start your study. And in reality, it takes anywhere from six months to nine months to even 12 months sometimes to get your study open at these sites, so patients can be enrolled, and there are lots of opportunities for AI to help us speed up that timeline, kind of globally. We’re starting to see that in protocol design and site selection feasibility, so that’s another area that I’m seeing kind of AI step into.
Bruno Quinney 06:32
So, you talk about the speed and efficiency of clinical trials because of AI being implemented. What are the other advantages? I know you touched upon a few there, but are there also, I suppose, financial advantages for companies using this technology?
Suzanne Caruso 06:46
Yeah, absolutely. So, I think the financial advantages are kind of twofold. I’ll give a use case on on speed, of course, equals equals money, but let’s use RWD. So, RWD, billions and billions of data points available. One of the things that we have seen for a long time is we believe that we have enough data in the public domain to build synthetic cohorts of patients, and what that means is historically we’ve always had a drug that’s active in a phase three and a placebo arm, which means we have patients who are just receiving essentially a sugar pill and not the active drug, so that you can have a control arm in any test you’re doing. Imagine being able to run an analysis using AI to match a patient population with a synthetic arm, so you don’t have to enrol those patients. What that means classically is that the study could decrease almost half the amount of time it takes to run, because you are enrolling half the amount of humans that you would have needed before. That is a massive, I mean, you’re talking about a million dollars a day to have a large phase three study open, that is a massive decrease in the amount of spend, and truthfully, more studies might be greenlit if those studies weren’t as expensive as they are, so that’s just an example of how AI is helping us run synthetic arms, therefore decreasing the amount of time that it takes to get to an outcome.
Bruno Quinney 08:13
I mean, you touched earlier upon maybe the mindset shift that’s needed to embrace AI. Do you think there is still a reticence with AI technology that maybe some of the results and some of the reliance and maybe trustworthiness of the technology isn’t always there.
Suzanne Caruso 08:29
Yes, yes, and yes. I mean, I don’t think any of us fully have seen something really wrong without hallucinations on an LLM, right? Like, hello. We all kind of got used to that, and I think our job now is to get the highest quality output by any AI agent or any model that we’re working with, and knowing that it’s always going to need some kind of oversight. Now, people are getting really sophisticated, and we’re even doing it here, which is, you know, you have agents now being monitored by agents that have a certain quality level, and you can decrease and decrease the amount of data quality deviations that you have, but there is always that kind of trust level, and trust is just going to come with time, and the other thing that I’ll say around trust, one of the things that’s been most important for us is our AI models, even the ones that we’re bringing to market, we have to cite sources. Sourcing is the most critical thing we can do in data intelligence. We’re doing it every day, so that any agent that we’re releasing to the market, we’re putting out a product called Atlas CI, it’s a competitive intelligence agent. We have to source every single data point that we have there to something that someone can reference outside of that agent, really, really important to build trust in that agent. So that’s another way that we’re kind of tackling this trust issue, which is what I think everyone has with new models.
Bruno Quinney 09:54
I suppose it’s one thing using AI to effectively run the trial, but then there’s another side of I. AI, which is being able to match patients with the right trials. So, could you talk to us a little bit about that, please? How is the technology in the models, how are they being used to match patients to the right clinical trials?
Suzanne Caruso 10:12
So, one of the things in clinical research is that you often, it’s not that there aren’t enough patients in the world, many trials we have enough patients in the world. It’s access to those patients, and where are those patients being seen? And that historically has, we’ve come to this with, okay, we’re going to choose HCPs that have been investigators previously to be able to have a site open, and we’re kind of passively waiting for them to show up at these hospitals, and then we’ll hopefully offer them an opportunity to be on a clinical research trial, and that really happens like only about 10% of the time. We have a way to be more proactive with AI, because we know where people are showing up that match the protocol criteria. Protocols might have 30 inclusion exclusion criteria. AI takes that text from a protocol, builds an algorithm that says these are the patients that would be eligible, and then looks at real-world data, which essentially patient data to say where are the patients that today meet the criteria for that inclusion exclusion criteria, that is something that has saved a tremendous amount of time in kind of patient matching your protocol to that patient population. Then the sponsor and the manufacturer can then go and say, I realise I have to go to Mount Sinai Hospital in New York City because they have the patient population that matches my protocol, and they have a consistent patient population and patient population flow that will match the protocol to be able to fully enrol it, so it’s just an area that AI has sped us along, instead of just kind of passively waiting for patients to come proactively, saying these are where the patients are, and taking a more proactive of it, so it’s kind of flipped the kind of paradigm of patient recruitment,
Bruno Quinney 12:01
so you’ve recruited patients for a trial, but then there’s also the question of how effective is the trial being delivered, and you know how effective is it being run.
Suzanne Caruso 12:10
Yeah, I think around this idea of effective is also quality, and the quality of the data that’s being put in in real time, and getting assessments on the data quality that’s being put in, and getting corrections almost immediately. There’s also kind of the effectiveness of how quickly those patients are being enrolled over time, and your ability to let the investigator know that there are additional patients that might be eligible, even that their colleagues have that they didn’t know about that, is something that AI is very good at, to be able to say your colleague, Dr. Smith, actually has a patient that might be eligible, you should reach out, because what that means is your study will enrol a little bit faster and be optimised a little bit more. The other piece is around monitoring, and I mentioned this a little bit earlier, which is we do a tremendous amount of monitoring of sites to make sure that the quality of the work that is being done is at a certain level. We are getting real-time feedback and the ability to adjust faster than ever, and that is absolutely being powered by the amount of data points that AI can go through in real time. Old days, it was like paper, then you got to kind of remote monitoring, but you still had the limit of the power of the quickly ability to quickly review data. Now that’s being done, you know, in minutes and seconds instead of days and weeks. So it’s just a different paradigm there.
Bruno Quinney 13:31
So I suppose we’re coming back to this question again of mentality and attitude towards AI, but if you run an organisation that’s slightly reticent towards AI, what are the low hanging fruit, and the very clear obstacles which AI can resolve very quickly.
Suzanne Caruso 13:47
One of the things I learned very quickly was that the best way to kind of engage with AI and figure out if it’s a good fit for you is to give it your problems and say how would you fix these problems, and I think a lot of organisations do struggle to adopt AI, and I understand why you have sites that you know, 50 sites that work on all different, you know, types of technology that sponsors are giving them to work with. How do you even stay up with the trends? I think part of that is just being open to the possibility that probably right now we are in this time of a boom of AI, a lot of different things being kind of tested and by individual sites, by individual sponsors, and we’ll kind of normalise on this on this trust level. The trust level in AI right now is much higher than it was previously, so working with AI every single day kind of helps you understand the value, and when there are still gaps, AI is not perfect, and I think we all need to acknowledge it is not perfect yet, but the more you use it, the more you figure out how to work and kind of minimise those gaps, whether on a clinical trial, just kind of be in your day-to-day work.
Bruno Quinney 14:58
I think you’ve said it yourself that AI. AI is not perfect yet, so are there any particular areas of clinical development where AI is not resolving everything as of yet?
Suzanne Caruso 15:07
Gosh, I’m sure there are like 400 areas where it’s not resolving everything just yet. One area that I, let’s see, would think about initially is study startup. When we’ve gotten to study startup, the ideal is we really can do study startup in like a month, month and a half. We are not there yet. I do think we’re making individual gains and individual pieces of study startup, but we have yet to see the efficiency from that entire workflow really kind of condensed down to a short period of time. So, protocol design, I think we’ve made a lot of headway there, as I said, site identification and selection, I think we’ve made a lot of head there, I think around patient engagement and recruitment, I think we’ve made a lot of headway there, but there are a lot of other components yet that have yet to kind of be, you know, optimised by AI, and I think we’re just picking off those pieces and kind of nascent in agents in those individual components.
Bruno Quinney 16:04
The more the technology and the models get developed, the quicker those problems are able to be overcome. So, just to draw the conversation to a close, is there one big takeaway that our listeners can take away from this episode in terms of how AI is transforming clinical workflows right now,
Suzanne Caruso 16:21
I think the one way, and the kind of one big takeaway is you have to try it. It is where we are all going. Don’t be scared of it, and know that it will always kind of need this human in the loop as well. And human in loop hasn’t been completely dissolved, and I don’t want people to think it has. We’ve actually done a tonne to optimise a lot of the low hanging fruit of the administrative burden to be able to allow you to supervise more, a lot of what the AI is doing. I also, we are seeing the efficiencies in enrollment and how quickly patients are being enrolled. We know that enrollment and patient identification has been a burden on sites for years and years, there just aren’t the people to be able to look through the massive amount of data. There’s one area that I would focus on that hopefully the sites could optimise, it is using AI to find patients, or at least reduce that 80% of the work that you do, so that they could be spending their time on the final qualifications of those patients, because patient identification just takes a tonne of time, so use it, lean into it, because I think it’s where we’re all going, and you’ll find out what areas are kind of like best situated for you.
Bruno Quinney 17:30
Suzanne, it’s been such a pleasure talking to you about this topic. I doubt it’s going to be the last time that we speak about the use of AI in clinical trials and in this industry, but thank you so much for giving us some insight into where we are right now.
Suzanne Caruso 17:44
Thank you so much for having me.
Bruno Quinney 17:48
That’s it for this week. Head to the DDW website for further information on these stories. Like and subscribe wherever you get your podcasts. If you’re enjoying this content, be sure to join free today to become a member of Drug Discovery World. Membership includes full access to the website, including free news and exclusive content, a weekly e-newsletter, podcasts, podcasts, white papers, e-books, and information from trusted third parties. See you next time.
