Summary: A study from Japan’s Osaka Metropolitan University found that while ChatGPT shows potential as a diagnostic tool in radiology, its accuracy falls short compared to board-certified radiologists, especially in musculoskeletal cases.

Key Takeaways

  1. ChatGPT shows potential in radiology diagnostics but requires further accuracy evaluation, particularly when compared to board-certified radiologists.
  2. GPT-4 outperformed GPT-4V and matched the performance of a radiology resident in analyzing musculoskeletal cases, highlighting its capabilities and limitations.
  3. The study emphasizes the need for a full understanding of ChatGPT’s diagnostic performance before it can be reliably used as a tool in radiology, despite its rapid advancements.

——————————————————————————————————————————————————

In radiology, diagnostic imaging requires specialized knowledge to interpret the findings associated with a wide variety of diseases. Recently, generative AI models like ChatGPT have shown potential as diagnostic tools, but their accuracy still needs thorough evaluation, according to a study in European Radiology.

Daisuke Horiuchi, MD, PhD, and associate professor Daiju Ueda, MD, from Osaka Metropolitan University in Japan led a study comparing ChatGPT’s diagnostic accuracy to that of radiologists. They analyzed 106 musculoskeletal radiology cases, including patient history, images, and findings.

Generative AI in Radiology

For the study, case data was entered into GPT-4 and GPT-4 with vision (GPT-4V) to generate diagnoses. Similarly, a radiology resident and a board-certified radiologist reviewed the same cases to provide their diagnoses. The results revealed that GPT-4 outperformed GPT-4V and matched the performance of the radiology resident. However, ChatGPT’s diagnostic accuracy fell short when compared to the board-certified radiologist.

“While the results of this study indicate that ChatGPT may be useful for diagnostic imaging, its accuracy cannot compare to a board-certified radiologist. Additionally, this study suggests that its performance as a diagnostic tool must be fully understood before it can be used,” says Horiuchi. “Generative AI, including ChatGPT, is advancing every day, and it is greatly expected to become an auxiliary tool for diagnostic imaging in the future.”