The authors evaluated Jev, nine configurations of Jev-like models, and 11 general-purpose or medical large language models across 15 datasets and four medical task families.
Medical decision models stay consistent but struggle to abstain reliably
Across 15 medical datasets, Jev was robust to question-format changes but weak on numerical tasks and unreliable at recognizing when no valid answer was available.
Chinese Tech
Yanrui Du · Danyang Zhao · Sendong Zhao · Wenhao Li · Yibo Gao · Haochun Wang · +3 more
Harbin Institute of Technology · Ant Group
Research Digest··2 min read
Du and colleagues compared 21 models on 8,742 medical examples using a system-one protocol, meaning models answered directly without producing reasoning traces.
§