By Vijay Kumar MalesuReviewed by Susha Cheriyedath, M.Sc.Jun 18 2024 In a recent study published in the new monthly journal NEJM AI, a group of researchers in the United States evaluated the utility of a Retrieval-Augmented Generation -enabled Generative Pre-trained Transformer -4 system in improving the accuracy, efficiency, and reliability of screening participants for clinical trials involving patients with symptomatic heart failure.
About the study In the present study, the Recurrent Error Correction with Tolerance for Input Variations and Efficient Regularization system was evaluated in the Co-Operative Program for Implementation of Optimal Therapy in Heart Failure trial, which compares two remote-care strategies for heart failure patients. Traditional cohort identification involved querying the EHR and manual chart reviews by non-clinically licensed staff to assess six inclusion and 17 exclusion criteria.
Fourteen prompts were used to generate "Yes" or "No" answers. Statistical analysis involved calculating sensitivity, specificity, and accuracy, with the Matthews correlation coefficient as the primary evaluation metric. Cost analysis and comparison across demographic groups were also performed. Overall, the sensitivity and specificity for determining eligibility were 90.1% and 83.6% for the study staff and 92.3% and 93.9% for RECTIFIER. When inclusion and exclusion questions were combined into two prompts or when GPT-3.5 was used instead of GPT-4 with the same RAG architecture, sensitivity and specificity decreased. Using GPT-4 without RAG for 35 patients, where 15 were misclassified by RECTIFIER for the symptomatic heart failure criterion, slightly improved accuracy from 57.1% to 62.
Law Law Latest News, Law Law Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Source: itvlondon - 🏆 116. / 51 Read more »