The study, titled "Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying," conducted by Ali Soroush et al., investigates the performance of various large language models (LLMs) including GPT-3.5, GPT-4, Gemini Pro, and Llama2-70b Chat in the task of medical code querying. The research demonstrates that these LLMs are generally ineffective at accurately generating medical billing codes such as ICD-9-CM, ICD-10-CM, and CPT from descriptions, with even the best performing model, GPT-4, failing to achieve high accuracy. Factors such as code frequency, brevity of code descriptions, and exactness of match were analyzed to understand performance disparities. The findings suggest that these LLMs, in their current state, are unreliable for medical coding tasks, often producing imprecise or entirely fabricated codes, which could undermine medical billing and record-keeping if used in clinical settings without further dedicated research and refinement.

Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying | NEJM AI NEJM AI

The article provides a comprehensive guide to various resources for individuals and businesses keen on integrating AI into their workflows. It includes a list of tools like ChatGPT+ and Hyperwrite AI for general and specific tasks, educational courses ranging from AI for Everyone to Practical Deep Learning for Coders, and several information channels such as newsletters (e.g., Ben’s Bites, AI Exchange) and podcasts (e.g., This Week in ML, AI Chat). Additionally, influential voices in AI, recommended books, community-endorsed tools, and startups are highlighted alongside top AI partners like AWS, Nvidia, and various AI consulting firms and investors, offering a detailed framework for anybody looking to deepen their understanding or expand their capability in AI technologies.

Top Tools for AI-First Workflows — Allie K. Miller alliekmiller.com

This article, authored by Justin G. Norden and Nirav R. Shah, explores artificial intelligence's role in health care and compares it to the development of autonomous vehicles (AVs). It suggests that AI in healthcare should initially support physician decision-making rather than aiming for full automation. Drawing parallels with the gradual advancement and current state of AV technology, where complete autonomy has not yet been achieved, the authors advocate a stepwise integration of AI in health care. They argue that while total replacement might work for vehicles, in health care, augmentation should be preferred over complete automation, recognizing the unique benefits and implications for patient care.

What AI in Health Care Can Learn from the Long Road to Autonomous Vehicles | NEJM Catalyst publication

Cybersecurity leadership in high-risk organizations demands not just technical skills but also personal qualities like mental toughness and perseverance, or what Angela Lee Duckworth defines as "grit." Duckworth outlines grit as being crucial for success, emphasizing the importance of passion, resilience, and dedication toward long-term goals over innate talent or intelligence. The field of cybersecurity, characterized by relentless threats and constant challenges, requires leaders to be resilient, adaptable, and continuously committed. To develop grit, cybersecurity leaders should engage in practices such as reflective scenario planning, setting and pursuing long-term goals, learning from setbacks, fueling their passion for the work, and routinely practicing perseverance and commitment. This development of grit is essential for leading effectively in the dynamic and demanding realm of cybersecurity, allowing leaders to inspire their teams and navigate the myriad of challenges with a sense of purpose and determination.

True Grit Substack

The study, titled "Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying," conducted by Ali Soroush et al., investigates the performance of various large language models (LLMs) including GPT-3.5, GPT-4, Gemini Pro, and Llama2-70b Chat in the task of medical code querying. The research demonstrates that these LLMs are generally ineffective at accurately generating medical billing codes such as ICD-9-CM, ICD-10-CM, and CPT from descriptions, with even the best performing model, GPT-4, failing to achieve high accuracy. Factors such as code frequency, brevity of code descriptions, and exactness of match were analyzed to understand performance disparities. The findings suggest that these LLMs, in their current state, are unreliable for medical coding tasks, often producing imprecise or entirely fabricated codes, which could undermine medical billing and record-keeping if used in clinical settings without further dedicated research and refinement.

Large Language Models Are Poor Medical Coders — Benchmarking of Medical Code Querying | NEJM AI NEJM AI

The article provides a comprehensive guide to various resources for individuals and businesses keen on integrating AI into their workflows. It includes a list of tools like ChatGPT+ and Hyperwrite AI for general and specific tasks, educational courses ranging from AI for Everyone to Practical Deep Learning for Coders, and several information channels such as newsletters (e.g., Ben’s Bites, AI Exchange) and podcasts (e.g., This Week in ML, AI Chat). Additionally, influential voices in AI, recommended books, community-endorsed tools, and startups are highlighted alongside top AI partners like AWS, Nvidia, and various AI consulting firms and investors, offering a detailed framework for anybody looking to deepen their understanding or expand their capability in AI technologies.

Top Tools for AI-First Workflows — Allie K. Miller alliekmiller.com

This article, authored by Justin G. Norden and Nirav R. Shah, explores artificial intelligence's role in health care and compares it to the development of autonomous vehicles (AVs). It suggests that AI in healthcare should initially support physician decision-making rather than aiming for full automation. Drawing parallels with the gradual advancement and current state of AV technology, where complete autonomy has not yet been achieved, the authors advocate a stepwise integration of AI in health care. They argue that while total replacement might work for vehicles, in health care, augmentation should be preferred over complete automation, recognizing the unique benefits and implications for patient care.

What AI in Health Care Can Learn from the Long Road to Autonomous Vehicles | NEJM Catalyst publication

Cybersecurity leadership in high-risk organizations demands not just technical skills but also personal qualities like mental toughness and perseverance, or what Angela Lee Duckworth defines as "grit." Duckworth outlines grit as being crucial for success, emphasizing the importance of passion, resilience, and dedication toward long-term goals over innate talent or intelligence. The field of cybersecurity, characterized by relentless threats and constant challenges, requires leaders to be resilient, adaptable, and continuously committed. To develop grit, cybersecurity leaders should engage in practices such as reflective scenario planning, setting and pursuing long-term goals, learning from setbacks, fueling their passion for the work, and routinely practicing perseverance and commitment. This development of grit is essential for leading effectively in the dynamic and demanding realm of cybersecurity, allowing leaders to inspire their teams and navigate the myriad of challenges with a sense of purpose and determination.

True Grit Substack