July 15, 2025Open Access

A Clinically-Informed Framework for Evaluating Vision-Language Models in Radiology Report Generation: Taxonomy of Errors and Risk-Aware Metric

View Full Paper

Ask AI

Bookmark

Proposed a clinically informed evaluation framework for vision-language models focused on radiology reports.
Defined a taxonomy of 12 radiology-specific error types, classified by clinical risk levels after consultation with physicians.
Conducted an error analysis on 685 MIMIC-CXR cases using three vision-language models highlighting critical vulnerabilities.
Introduced a clinical risk-weighted error score (CREST) metric to assess model safety and improve report generation.

Authors

Discussion

No takes yet. Share an insight, caveat, or question.

Clinical evaluation framework improves vision-language models for radiology report generation, indicating critical error patterns and risks involved.

Guan et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark

Clinical evaluation framework improves vision-language models for radiology report generation, indicating critical error patterns and risks involved.