Can AI Reason Like a Clinician? An Exploration with Arterial Blood Gas Analyses

This article discusses using generative AI models like Anthropic Claude v2 to assist clinicians in interpreting arterial blood gas (ABG) results and diagnosing acid-base disorders. It explores the strengths and weaknesses of large language models (LLMs) for this task, and techniques like prompt sculpting, retrieval-augmented generation, and using a "math scratchpad" to improve the accuracy of the LLM's output.

Specifically, the article covers:

• Background on using LLMs for healthcare applications and challenges like hallucination and mathematical limitations
• Using Amazon Bedrock to deploy LLMs for ABG interpretation
• Techniques to improve the accuracy of LLM outputs, including prompt sculpting, retrieval-augmented generation, and using Python for mathematical calculations
• Results showing improved accuracy (over 85%) after applying these techniques
• Conclusion on the promise of LLMs for assisting clinicians, with techniques to improve accuracy