The authors created JEVal, a bilingual benchmark containing 11,257 instances drawn from 36 datasets and 10 application domains.
Fast decision models excel locally but falter in extended systems
Across bilingual benchmarks, agent tasks and social simulations, specialized decision models were efficient but accumulated errors and overstated confidence.
Top University
Feiyu Duan · Jiayu Lin · Jia Wang · Jun Xiang · Jialiang Wu · Xinnong Zhang · +3 more
Shanghai Innovation Institute · Fudan University · Tongji University · King’s College London · Chinese University of Hong Kong
Research Digest··3 min read
Duan, Lin and colleagues evaluated 25 configurations of general decision models and generative language models, then tested how their local predictions behaved inside longer workflows.
§