The authors tested models on MMLU, a multiple-choice benchmark spanning academic and professional subjects.
Reasoning models follow unreliable hints in a predictable mathematical pattern
Across three model families, the authors find structured shifts toward suggested answers, even when prompts explicitly warn that the source is unreliable.
Big Tech
Anxin Guo · Chirag Pabbaraju · Sara Ahmadian · Kamesh Munagala · Aravindan Vijayaraghavan
Northwestern University · NSF-Simons AI Institute for the Sky (SkAI) · Stanford University · Google Research
Research Digest··2 min read
Guo and colleagues measure how unsupported hints change reasoning models’ answers to multiple-choice questions.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.