Search

20 results for “MMLU”

1. Research · Agents

   Draft-KV ensures the receiver actually uses the sender's latent message

   Draft-KV: Learning Useful Latent Communication Between Language Models

   By sending key-value states from the sharer's draft, a lightweight interface boosts a 0.5B receiver to 78% on MMLU-Redux, and message-replacement tests confirm the gain comes from genuine information use.

   MMLU-Redux: 78.04% accuracy (as reported)

   29 Sept 2026City University of Hong Kong, University of Science and Technology of China, University of Electronic Science and Technology of China

2. Research · Language models

   Query-adaptive neuron steering improves frozen language models across benchmarks

   Query-Adaptive Neuron Steering for LLMs

   Q-NeuS uses each query’s internal activation pattern to select and scale neuron interventions without updating the underlying model.

   MMLU-Pro: 48.53% accuracy (as reported)

   15 Oct 2026Northeastern University, Washington University in St. Louis, Nanjing University of Aeronautics and Astronautics

3. Research · Reinforcement learning

   Turning failed reasoning attempts into training signal improves GRPO on hard problems

   Failure Is the Mother of Success: Recovering Supervision with Grounding Hindsight

   RSHD generates corrective notes from failures, verifies them via replays, and distills successful behaviors into the policy without extra inference cost.

   MMLU-Pro: 61.2% accuracy (as reported)

   16 Oct 2026University of California, San Diego, Adobe Research, University of Virginia

4. Research · Agents

   Progressive pruning lets language-model debates beat equal-cost baselines

   Beyond Solo and Consistency: Vindicating Multi-Agent Debate via Conditional Progressive Pruning

   Conditional Progressive Pruning uses intermediate debate results and learned agent behavior to allocate computation across rounds more effectively.

   MMLU-Pro: 68% accuracy (as reported)

   5 Oct 2026Rutgers University, New Brunswick, University of Cambridge, Tsinghua University

5. Research · Efficiency & systems

   Multi-precision LLM weight format keeps a single prefix-readable stream

   StagQ: Constraint-Driven Multi-Precision Weight Quantization for LLMs

   StagQ stores weights as a 2-bit base with refinement planes, using a side record for outliers, and outperforms baselines on MMLU at 2 and 3 bits.

   MMLU: 64.11% accuracy (as reported)

   8 Oct 2026Huawei Technologies Ltd.

6. Research · Language models

   Role prompts help language models only under the right conditions

   Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models

   Across multilingual knowledge benchmarks, persona prompting varied with model capacity, subject and language, while combining equivalent prompts across languages produced more consistent results.

   MMLU: 0.59 accuracy (as reported)

   7 Oct 2026Xiamen University, Tongyi Lab, The Chinese University of Hong Kong

7. Research · Agents

   Revision-aware agent graphs reuse valid work while avoiding stale answers

   Revision-Aware Independent Agent Graphs for Dynamic Reasoning

   RIAG combines deterministic version selection, version-keyed caching and bounded independent reasoning to handle tasks revised over time.

   Full collection (6 benchmarks): 54.24% joint accuracy (as reported)

   2 Oct 2026Harvard AI and Robotics Lab, Harvard University

8. Research · Language models

   Alternating context edits and logit steering boosts frozen LLM accuracy

   EMO: A Closed-Loop Test-Time Computation Framework via Alternating Context Optimization and Logits Steering

   EMO closes the loop between feedback-driven context revision and decoding-time distribution correction, improving zero-shot performance across eight models and six benchmarks.

   MMLU-Pro: 9.6 pts oracle coverage improvement over TextGrad (as reported)

   16 Oct 2026Peking University, Samsung Research

9. Research · Reinforcement learning

   Posterior rewards become indistinguishable for long reasoning traces in LLM training

   Rethinking Probability-Based Reinforcement Learning From Posterior Concentration

   The authors identify a length-dependent failure mode they call Posterior Concentration Phenomenon, and propose a framework to mitigate it.

   MMLU-Pro: 61.3% Avg@2 (as reported)

   7 Oct 2026National University of Singapore, The Hong Kong University of Science and Technology, University of Science and Technology of China

10. Research · Language models

    AI chatbots fail long tasks due to competing content, not length

    Length Is Cheap, Competition Is Not: Attention Predicts Context Rot in Retrieval

    Neutral filler costs nothing in retrieval tests, while a few rival statements crash accuracy and attention on the evidence predicts the loss.

    MMLU: 65.8% accuracy (as reported)

    4 Nov 2026University of California San DiegoCode

11. Research · Efficiency & systems

    Tetra serves 2.7-bit language models without expanding lattice codes

    Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter

    A fused GPU kernel decodes compact Leech-lattice weights during multiplication, reducing memory traffic while retaining usable accuracy on Qwen3 models.

    MMLU: 63.37% accuracy (as reported)

    5 Oct 2026

12. Research · Safety & security

    Method to permanently erase unwanted LLM personas via weight edits

    LLM Persona Unlearning

    A new benchmark and contrastive erasure technique suppress target personas while preserving utility across six models and five personas.

    PersonaUnlearnBench: 100% TF (as reported)

    1 Oct 2026University of Macau, Mila – Québec AI Institute, McGill University

13. Research · Evaluation & benchmarks

    Current date in system prompts skews LLM evaluation results

    Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation

    The authors show that LLM accuracy varies by up to 14% on math reasoning and reshuffles leaderboards when only the date changes, making the effect larger than other known sources of non-determinism.

    MMLU: 2.3% worst-to-best date delta in accuracy (as reported)

    6 Oct 2026Johannes Gutenberg University Mainz, Universidad Iberoamericana, University of Colorado Boulder

14. Research · Language models

    Language models improve using self-generated questions and answers, without human examples

    On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models

    Sick and colleagues report gains in language and visual reasoning using self-generated supervision rather than human-curated training questions or reference answers.

    MMLU: 61.9% accuracy (as reported)

    26 Oct 2026Google, Ulm University

15. Research · Language models

    Language models retrieve information from long texts better with wider attention heads

    Head Dimension Matters in Language Models for Long-Context Retrieval

    Under a fixed parameter budget, models with fewer but wider heads outperform those with narrower heads on retrieval tasks.

    RULER-100: 36.72% overall score (as reported)

    26 Oct 2026Preferred Networks, Inc.

16. Research · Efficiency & systems

    Language models can be trained for compression nearly 13 times faster

    SQuAT: Spherical Lattice Vector Quantization-Aware Training

    Loeschcke et al. decompose weight blocks into direction and magnitude and apply factored straight-through estimators, improving accuracy over joint-STE baselines and matching scalar INT-QAT below 2.5 bits per weight.

    MMLU: 69.5% accuracy (as reported)

    23 Oct 2026University of Copenhagen, Qualcomm AI Research

17. Research · Safety & security

    Pretrained LLMs already contain refusal circuits that shift during alignment

    SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models

    SafeEvo identifies sparse refusal circuits in base models, traces their structural drift across safety checkpoints, and uses them to improve fine-tuning.

    HarmBench: 2.29 pts average ASR reduction (as reported)

    9 Oct 2026The University of Hong Kong, Chinese Academy of Sciences, Information Engineering University

18. Research · Language models

    Bi-encoders can support multimodal decisions across small and massive choice sets

    MetaEncoder: Exploring the Limit of Bi-Encoders for Multimodal System One Decision Making with Natural Language Interface

    MetaEncoder uses one natural-language interface to score options in closed sets and retrieve items from collections containing millions of candidates.

    MMLU: 0.78 Hit@1 (as reported)

    10 Oct 2026Meta AI

19. Research · Language models

    Decoding as optimisation on the probability simplex unifies and composes sampling strategies

    Composable Decoding on the Probability Simplex: Theory and Implementation

    The authors formalise decoding as a regularised optimisation problem, recovering standard methods as special cases and enabling new decoders through compositional regularisers.

    GPQA Diamond: 26.3% pass@1 (as reported)

    5 Oct 2026Huawei Noah's Ark Lab, UCL Centre for AI

20. Research · Language models

    Routing-signature diversity prevents expert collapse and boosts MoE specialization

    Function Over Form: Distributional Orthogonalization in Mixture-of-Experts with Replica Expert Mechanism

    The authors introduce DO-loss to penalize overlapping token assignments and REM to rebalance workloads without sacrificing specialization.

    MMLU: 54.96% accuracy (as reported)

    4 Oct 2026Peking University, Tencent Hunyuan