Search

2 results for “MMLU-PRO”

1. Research · Evaluation & benchmarks

   Language models’ own confidence becomes harder to outperform as they improve

   Predictable performance is suboptimal, and what it means for model routing

   Zhang and Hardt connect prediction gains beyond a model’s own confidence to its room for improvement, then test the relationship across 16 language models.

   MMLU-Pro: 0.65 Pearson correlation between empirical suboptimality and (as reported)

   23 Oct 2026

2. Research · Agents

   Joint harness search balances model accuracy, safety, and token use

   MoMHa: Multi-Objective Optimization of LLM Harnesses over Accuracy, Safety, and Tokens

   Across 17 domains and 12 models, jointly optimizing the code around language models outperformed sequential and accuracy-only approaches.

   3D hypervolume: 0.48 hypervolume indicator (as reported)

   28 Sept 2026Adobe Research