Search

3 results for “MMLU-PRO”

1. Research · Evaluation & benchmarks

   Scaling curves can be certified with far fewer sampled answers

   Cheap to Draw, Expensive to Trust: Certifying Test-Time Scaling Curves

   A variance-aware audit provides simultaneous confidence bands across test-time budgets while reducing the number of generated answers.

   MMLU-Pro: 79133 generated answers (as reported)

   7 Oct 2026PMCC AI Lab, Peter Munk Cardiac Centre, University Health Network

2. Research · Evaluation & benchmarks

   Language models’ own confidence becomes harder to outperform as they improve

   Predictable performance is suboptimal, and what it means for model routing

   Zhang and Hardt connect prediction gains beyond a model’s own confidence to its room for improvement, then test the relationship across 16 language models.

   MMLU-Pro: 0.82 Pearson correlation between empirical suboptimality and (as reported)

   23 Oct 2026

3. Research · Agents

   Revision-aware agent graphs reuse valid work while avoiding stale answers

   Revision-Aware Independent Agent Graphs for Dynamic Reasoning

   RIAG combines deterministic version selection, version-keyed caching and bounded independent reasoning to handle tasks revised over time.

   Full collection (6 benchmarks): 0.62 calls/query calls per query (as reported)

   2 Oct 2026Harvard AI and Robotics Lab, Harvard University