California-based Mendel.ai has announced that a study utilising its artificial intelligence (AI) software alongside nurses has shown more accurate and quicker pre-screening of oncology patients for clinical trials.
Mendel.ai, who conducted the trial in collaboration with the University of Pennsylvania, announced the data at the American Society of Clinical Oncologists (ASCO) Meeting 2024, in abstract 1524, “Human-AI teams to improve accuracy and timeliness of oncology trial pre-screening: Preplanned interim analysis of a randomized trial.”
Go deeper with GlobalData
The trial investigated the use of the AI-augmented software alongside a human (human and AI) for clinical data extraction to determine eligibility against the traditional workflow of a human alone.
The trial evaluated unstructured, real-world data, with results showing that there was evidence of both improved accuracy and timeliness when incorporating AI into the task for cancer data extraction.
Among 74 patients evaluated, accuracy for human plus AI was noninferior to human alone (78.7% vs. 76.7%) and both were greater than AI-alone (63.5%).
Median time per review was also lower for human plus AI at 34.1 minutes per case than human-alone at 43.9 minutes.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataKarim Galil, MD, co-founder, and CEO of Mendel.ai, which developed the natural language processing software used for the study, spoke exclusively to the Clinical Trials Arena about how the software works and how it will integrate with current electronic medical records.
This interview has been edited for length and clarity.
Abigail Beaney: How does the human plus AI system work to support pre-screening for these patients?
Karim Galil: AI should be all about co-pilots, which is augmentation of humans, rather than replacement of humans. The goal here was, can the AI help nurses in chart review to determine whether a patient is eligible or not for a trial?
As you can imagine, an oncology patient has hundreds of pages of records so it can take nurses a long time to go through. We have picked the data elements you would associate with oncology trial eligibility like biomarkers, staging, type of cancer. We have then indexed all of that with our AI software so the nurse can log in and quickly navigate through the patients notes and decide whether the patient is eligible. We found that not only were they significantly faster at doing this than a human alone, but it was no less accurate than if they had done it themselves.
AB: Why can the AI not complete this task without human interaction?
KG: AI systems today have a few problems such as lack of reasoning, hallucination and lack of explainability.
Lack of reasoning for example, emergency room will be shortened to ER however there is also the biomarker of oestrogen receptor which is also shortened to ER. A lot of the AI models today, specifically large language models, would not be able to differentiate and that is lack of reasoning – the AI is not mimicking how a physician will understand the context of the sentence. That takes us onto hallucination – sometimes the AI would take ER as a biomarker and sometimes emergency room which is incredibly dangerous. If a system is consistently wrong, that’s better than a system that sometimes is wrong because as a human, you will start building levels of trust. Finally, these models do not show how it made such a conclusion. When the nurse is working with the AI, it is showing the evidence it sourced to come to such a conclusion so the nurse can use their judgement as to whether it is relevant or not.
AB: How does this software looks for the nurse completing the pre-screening process?
KG: When you log into the interface, you essentially see the actual medical record of the patient, but the application programming interface (API) directs you the relevant information which might make the patient eligible for the trial. The nurse can then review that data and decide whether it is correct and relevant.
AB: How does this integrate with already installed electronic medical records (EMRs) in hospitals?
KG: Building an AI is a big challenge in itself but making it usable is an even bigger challenge. With this, it needs to work with all different EMR systems while also considering privacy laws. Essentially, you want the system to be sitting in your hospital, but you don’t want it to hold the data in open AI. This software integrates with almost every system out there as the system stores the data on the hospital-side, so we also don’t need to consider informed consent.
