Search

66 results for “GSM8K” · page 2 of 4

1. Research · Evaluation & benchmarks

   Current date in system prompts skews LLM evaluation results

   Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation

   The authors show that LLM accuracy varies by up to 14% on math reasoning and reshuffles leaderboards when only the date changes, making the effect larger than other known sources of non-determinism.

   GSM8K: 7.75% worst-to-best date delta in accuracy (as reported)

   6 Oct 2026Johannes Gutenberg University Mainz, Universidad Iberoamericana, University of Colorado Boulder

2. Research · Efficiency & systems

   Random Outputs Can Replace Model Updates in Collaborative Learning

   Distributed Subliminal Learning: Replacing Model Updates with Random-Carrier Outputs

   The authors distill task-unrelated model outputs to combine specialized language models and train federated classifiers with substantially less uplink communication.

   GSM8K: 32.2% accuracy (as reported)

   8 Oct 2026Università della Svizzera italiana (USI), MIT Media Lab, Massachusetts Institute of Technology

3. Research · Evaluation & benchmarks

   Pass@k misses output diversity changes caused by model post-training

   What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-Training

   GRPO and rejection-sampling fine-tuning produced sharply different output distributions despite often similar pass@k scores.

   GSM8K: 0.97 pass@32 (as reported)

   8 Oct 2026Accenture Strategy & Consulting, Vizuara AI Labs

4. Research · Safety & security

   Private-query AI assistants leak secrets through rejected drafts, new protocol plugs the hole

   Answer-Only Disclosure: Private-Context LLM Assistance over Web APIs with a Leakage Ledger

   A protocol called ADLER ensures the cloud sees only what the user sees, and a leakage ledger caps how much the answer reveals about private documents

   GSM8K: 5.4 s modelled communication time per answer (as reported)

   30 Oct 2026Australian Catholic University, Ant Group, University of Macau

5. Research · Evaluation & benchmarks

   Reused evaluation sets exaggerate gains in LLM self-improvement loops

   The Winner's Curse in LLM Self-Improvement Loops: Selection Noise, Lock-in, and Acceptance Rules

   Experiments with prompt-rewriting systems show that repeatedly selecting candidates on small evaluation sets creates large gaps between reported and held-out performance.

   GSM8K: 19.6 pts proxy-held-out gap (as reported)

   8 Oct 2026Meta, Microsoft

6. Research · Efficiency & systems

   Two-rate precision allocation cuts MoE memory footprint and decode traffic

   DuoRate: Two-Rate Per-Expert Precision Allocation for Mixture-of-Experts Serving

   DuoRate stores weights once and adapts read precision per workload, matching 4-bit accuracy at 2.9 bpw

   GSM8K: 88.25% accuracy (as reported)

   12 Oct 2026Huawei Technologies Ltd.

7. Research · Language models

   Language models improve using self-generated questions and answers, without human examples

   On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models

   Sick and colleagues report gains in language and visual reasoning using self-generated supervision rather than human-curated training questions or reference answers.

   GSM8K: 80.9% accuracy (as reported)

   26 Oct 2026Google, Ulm University

8. Research · Language models

   Language models retrieve information from long texts better with wider attention heads

   Head Dimension Matters in Language Models for Long-Context Retrieval

   Under a fixed parameter budget, models with fewer but wider heads outperform those with narrower heads on retrieval tasks.

   RULER-100: 31.25% overall score (as reported)

   26 Oct 2026Preferred Networks, Inc.

9. Research · Language models

   Shaping teacher behavior improves self-distillation for language models

   TeachBack: Learning to Teach via Bidirectional Preference Optimization for On-Policy Self-Distillation

   The TeachBack framework uses bidirectional preference optimization to encourage teachers to produce exploration-rich reasoning trajectories that students can learn from.

   AMC23/AIME24/AIME25: 76.29% mean@4 (as reported)

   22 Oct 2026Huazhong University of Science and Technology, Tsinghua University, Harbin Institute of Technology

10. Research · Evaluation & benchmarks

    Language models’ own confidence becomes harder to outperform as they improve

    Predictable performance is suboptimal, and what it means for model routing

    Zhang and Hardt connect prediction gains beyond a model’s own confidence to its room for improvement, then test the relationship across 16 language models.

    GSM8K: 0.93 Pearson correlation between empirical suboptimality and (as reported)

    23 Oct 2026

11. Research · Language models

    Gradient decomposition supplements LoRA updates to match full fine-tuning performance

    Beyond Low-Rank Parameterization: Narrowing the Gap Between LoRA and Full Fine-Tuning via Gradient Decomposition

    GDLoRA recovers gradients orthogonal to LoRA's accessible space and applies them to base weights, improving accuracy across multiple benchmarks by 2–4%.

    GLUE: 85.26% average score (as reported)

    4 Oct 2026Huazhong University of Science and Technology, Hebei University of Technology

12. Research · Efficiency & systems

    Language models can give identical answers up to 2.78 times faster

    ValuSpec: Plug-and-Play Candidate Valuation before Target Verification for Tree-Based Speculative Decoding

    The plug-and-play module uses transfer-based supervision to align filtering with the target model without training.

    GSM8K: 98.25 tokens/s throughput (as reported)

    23 Oct 2026Tongji University, Nanyang Technological University, Marquette University

13. Research · Efficiency & systems

    Lower precision beats structured pruning until GPU quantization limits are reached

    Is 2:4 Sparsity Still Worth It? Semi-Structured Sparsity versus Quantization in LLM Inference

    Across three NVIDIA GPU generations, Qin finds that 2:4 sparsity becomes useful chiefly when no lower dense precision remains available.

    GSM8K: 0.31 accuracy (as reported)

    22 Oct 2026Code

14. Research · Language models

    Progressive verification speeds up diffusion language model inference

    PV-Decoding: Progressive Verification Decoding with Decode Gating for Diffusion Language Models

    PV-Decoding uses confidence thresholds and progressive verification to reduce model forward passes by 8–37% while keeping accuracy drop under 1.3%.

    GSM8K: 68.5 MFP (as reported)

    21 Oct 2026Huawei Technologies, City University of Hong Kong

15. Research · Safety & security

    LookAhead Defense uses the model's own safe behavior to block dormant poisoning.

    Defending Against Dormant Poisoning Attacks Across Language and Multimodal Agents

    It penalizes only updates that weaken safety, reducing harmful agent actions from 93.9% to 14.2%.

    ASB: 7% poison activation (as reported)

    20 Oct 2026UNC Chapel Hill, Kyung Hee University

16. Research · Language models

    A learned router chooses when language models should draft farther ahead

    Next Token or Vector? Learning How Far to Decode

    CADENCE predicts variable-length token spans as latent vectors, selecting longer drafts only when the current context appears sufficiently predictable.

    GSM8K: 51.7% token acceptance rate (as reported)

    19 Oct 2026Microsoft, University of Massachusetts Amherst

17. Research · Efficiency & systems

    Speculative decoding throughput gains are conditional on workload and serving conditions

    When Does Better Acceptance Pay? A Controlled Study of Speculative Decoding

    A controlled study across six drafter recipes and ten benchmarks shows that aligning training and inference conditions narrows method gaps, and acceptance improvements only translate to higher throughput under specific task uncertainty and execution contexts.

    GSM8K: 5.44 tokens/s speedup (as reported)

    20 Oct 2026Tianjin University of Technology, Baidu Inc., Shandong University

18. Research · Language models

    Computational graphs replace free-form reasoning for verifiable LLM outputs

    Structural Chain-of-Thought: Reasoning with Executable Computational Graphs

    The Structural Chain-of-Thought method trains models to generate executable directed acyclic graphs, improving accuracy, efficiency, and faithfulness across multiple benchmarks.

    GSM8k+: 0.35 accuracy (as reported)

    15 Oct 2026MIPT, AIRI

19. Research · Language models

    On-policy distillation can transfer think-mode advantage to no-think responses via trace-response routing

    Learning from Think-Mode Advantage via On-Policy Distillation

    Authors propose ThinkOPD, which ranks student responses by outcome benefit and trace-response compatibility, outperforming uniform distillation and baselines on math and code tasks.

    AIME24: 35.2% avg@16 (as reported)

    6 Oct 2026

20. Research · Language models

    Coarse-to-fine sparse attention boosts long-context LLM inference speed

    Prism Attention: Proposal-Refined Index Sharing Mechanism for Efficient LLMs Inference

    Prism Attention replaces static cross-layer index sharing with dynamic refinement, achieving 3.4x speedup over FlashAttention at 128K tokens.

    128K context: 3.36 x end-to-end speedup over FlashAttention (as reported)

    19 Oct 2026Soochow University, Baidu Inc.