September 25, 2025Open Access

Exploration of Stability Judgments: From Multimodal LLMs to Human Insights

View Full Paper

Ask AI

Bookmark

Human participants achieved the highest accuracy in reasoning tasks, showcasing a significant gap between humans and models.
Among models, GPT-4o excelled in image-based tasks, while Qwen2.5VL achieved top scores overall, sometimes surpassing commercial models.
Binary tasks showed balanced performance for models, but accuracy declined sharply in complex multiple-choice tasks.
Structured inputs like XML improved prediction tasks; findings highlight advances in model design and performance in physical reasoning.

Discussion

No takes yet. Share an insight, caveat, or question.

This exploration evaluates multimodal large language models' accuracy in physical reasoning, revealing gaps compared to human participants.

Dewantoro et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark

This exploration evaluates multimodal large language models' accuracy in physical reasoning, revealing gaps compared to human participants.