Search

8 results for “GSM8K”

1. Research · Language models

   Language models given hints make larger models’ performance more predictable

   Hinted Scaling Laws

   The authors report that using assisted answers from smaller models reduced median error in forecasts of larger models’ unaided accuracy from 21 to 4.3 percentage points.

   GSM8K: 3 pts forecasting error (as reported)

   3 Nov 2026Stanford University

2. Research · Theory

   Random weight perturbations don't reliably find task experts in pretrained models

   Does Random Search Discover Task Experts? Revisiting Neural Thickets

   The apparent advantage of best-of-N selected perturbations mostly reflects selection noise, and top experts perform no better than the base model on held-out questions.

   Countdown, GSM8K, MBPP: 0.16 pts mean held-out rank-1 accuracy gain (as reported)

   3 Nov 2026University of Southern California, University of Oxford

3. Research · Efficiency & systems

   4-bit language models run as integer math on Apple's M5 chips without extra accuracy loss

   Integer-Exact Microscaling: Running MXFP4 Language Models on INT8 Matrix Units

   The authors show MXFP4 weights can be folded into a single INT8 matrix multiply when their block exponents span at most three, a condition met by over 97.9% of rows in real models.

   GSM8K: 94.8% accuracy (as reported)

   3 Nov 2026UT Southwestern Medical Center, Emory University, Duke UniversityCode

4. Research · Reinforcement learning

   Language models can improve answers without better internal error recognition

   RLVR Sharpens the Pencil, Not the Mind

   Tracking one small model through arithmetic training, the authors find modest accuracy gains but no measurable improvement in internal signals distinguishing flawed reasoning.

   GSM8K: 0.62 pass@1 (as reported)

   3 Nov 2026Indian Institute of Technology Roorkee

5. Research · Reinforcement learning

   Reasoning models can be trained to stop when they know the answer

   Think Less, Locally: Local Preference Learning for Reasoning Compression

   A new method teaches large language models when to conclude reasoning rather than continue, cutting output length by up to 45% on math and 36% on code without losing accuracy.

   AIME24: 7364 Tokens (as reported)

   4 Nov 2026Xi’an Jiaotong University, Tencent Inc.

6. Research · Language models

   Prompt optimization yields no gains on strong reasoning models

   Compile as Hypothesis: A Headroom Typology for Prompt Optimization on Real Reasoning Models

   Automated prompt compilers show no statistically significant accuracy improvements across four optimizer families and eight tasks, but structured genomes prevent catastrophic search failures.

   AIME 2024+25: 60 problems correct (as reported)

   3 Nov 2026Hunan Agricultural University

7. Research · Efficiency & systems

   Pruned language models retain problem-solving coverage despite lower one-shot accuracy

   Rethinking LLM Pruning via Test-Time Scaling

   Experiments on math and coding benchmarks show that structured pruning preserves the range of solvable problems even as single-attempt success declines, but obtaining those solutions requires more sampling effort.

   four tasks: 18.8% coverage retention (as reported)

   3 Nov 2026Tencent

8. Research · Safety & security

   Private-query AI assistants leak secrets through rejected drafts, new protocol plugs the hole

   Answer-Only Disclosure: Private-Context LLM Assistance over Web APIs with a Leakage Ledger

   A protocol called ADLER ensures the cloud sees only what the user sees, and a leakage ledger caps how much the answer reveals about private documents

   GSM8K: 0.83 lenient exact match accuracy (as reported)

   30 Oct 2026Australian Catholic University, Ant Group, University of Macau