July 28, 2025

Performance Evaluation of Large Language Models in Multilingual Medical Multiple-Choice Questions: A Mixed Method Approach (Preprint)

View Full Paper

Ask AI

Bookmark

Authors

Discussion

Mixed-method approach reveals performance variability in multilingual medical questions, implying careful model selection is needed.

Large language models exhibit variable performance across three languages in Swiss medical multiple-choice questions.
Quantitative and qualitative analyses reveal that while many questions are answered correctly, accuracy can differ significantly by model.
Most models demonstrated reasoning errors, struggling with identifying the best answers despite showing factual accuracy.
The study emphasizes the need for ongoing evaluation and transparent reporting to integrate AI effectively in medical education.

Strasser et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark