Search

12 results for “MATH-500”

1. Research · Efficiency & systems

   Scalar two-bit weight quantization can preserve language model reasoning accuracy

   ThinkBit: Search Codewords, Recalibrate Scales, Select by Reasoning

   ThinkBit, a five-stage framework, keeps MATH-500 accuracy within a few points of bfloat16 on models up to 14B while shrinking memory traffic by up to 8x.

   MATH-500: 47.5% accuracy (as reported)

   27 Oct 2026Microsoft Applied Sciences Group

2. Research · Language models

   Starting token cues can elicit reasoning from base language models

   Base Models Can Reason By Taking a Cue From Training Data

   Fixing just the first two tokens of a response dramatically improves performance on math and coding, rivaling reinforcement learning.

   MATH-500: 16.5% pass@1 (as reported)

   6 Oct 2026

3. Research · Language models

   Training on self-generated answers improves reasoning in low-bit language models

   Right On Q: Recovering Reasoning under Quantization via Self-Generated Trajectories

   Across three Qwen3 model sizes at 2- and 3-bit precision, OnQ improves benchmark accuracy while reducing truncated and repetitive responses.

   MATH-500: 69.33% accuracy (as reported)

   22 Oct 2026University of Michigan, Ann Arbor, University of California, Santa Barbara, Meta AI

4. Research · Language models

   Sparse RL-Guided Token Choices Recover Most Math Reasoning Gains

   What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make

   Replacing about 4% of a base model’s generated tokens with an RL-trained model’s choices recovered 90.5% of the reinforcement-learning improvement on AIME.

   AIME 2022-2025: 18.44% pass@1 (as reported)

   20 Oct 2026University of California, San Diego, Meituan, LongCat Team, The Chinese University of Hong Kong, Shenzhen

5. Research · Language models

   Cross-feedback and step-wise scoring let small models match much larger reasoning datasets

   Collaborative Reasoning Distillation via Cross-Feedback and Coherent Curation

   The authors' CRD framework iteratively critiques multiple teachers, scores individual reasoning steps, and stitches complementary chains to train compact models on 50K examples.

   MATH-500: 91.1% pass@1 (as reported)

   9 Oct 2026KAIST

6. Research · Efficiency & systems

   Language models can adapt compressed memory without growing their dictionaries

   FAD-KV: Adaptive Dictionary KV Caching for Efficient Reasoning Decoding

   The authors combine faster selection of compression components with reuse of inactive components, keeping the dictionary fixed in size as generation continues.

   MATH-500: 85.7% full-OMP support recall (as reported)

   30 Oct 2026Rensselaer Polytechnic Institute, IBM Research

7. Research · Language models

   Language models gain from varied answers, not necessarily altered weights

   Rethinking Neural Thickets Without Weight Perturbations

   Dufour and colleagues show that changing output scores, selecting useful variants and combining their answers can match or exceed ensembles built by perturbing model weights.

   MATH-500: 81.3% accuracy (as reported)

   30 Oct 2026Kyutai, Meta FAIR, LIGM

8. Research · Language models

   Reasoning models usually inherit “wait” from training data, not reinforcement learning

   Inherited, Not Invented: Tracing the Provenance of Wait in Reasoning Models

   Across open-model training histories, the authors find that reinforcement learning can improve mathematics accuracy without introducing “wait”, while different training recipes amplify or erase an inherited marker.

   MATH-500: 91.4% accuracy (as reported)

   23 Oct 2026The Hong Kong University of Science and Technology, Huawei Noahah’s Ark Lab, Peking University

9. Research · Language models

   Uncertainty-weighted credit improves cross-tokenizer distillation for LLMs

   CAST: Credit Alignment for Cross-Tokenizer On-Policy Distillation

   Authors' CAST framework redistributes chunk-level teacher advantages across student tokens and anchors unaligned positions, outperforming prior baselines on math, science, and code benchmarks.

   six-benchmark average: 47.22% pass@1 (as reported)

   12 Oct 2026Taobao & Tmall Group of Alibaba, Singapore Management University

10. Research · Evaluation & benchmarks

    Pass@k misses output diversity changes caused by model post-training

    What pass@k Cannot Measure: Evaluating Diversity and Capability Retention after Post-Training

    GRPO and rejection-sampling fine-tuning produced sharply different output distributions despite often similar pass@k scores.

    GSM8K: 0.63 pass@1 (as reported)

    8 Oct 2026Accenture Strategy & Consulting, Vizuara AI Labs

11. Research · Reinforcement learning

    Posterior rewards become indistinguishable for long reasoning traces in LLM training

    Rethinking Probability-Based Reinforcement Learning From Posterior Concentration

    The authors identify a length-dependent failure mode they call Posterior Concentration Phenomenon, and propose a framework to mitigate it.

    MATH-500: 82.9% Avg@2 (as reported)

    7 Oct 2026National University of Singapore, The Hong Kong University of Science and Technology, University of Science and Technology of China

12. Research · Reinforcement learning

    Preserving multiple reasoning strategies prevents late-stage collapse in RLVR

    All Work And No Play Makes Jack a Dull Boy: Understanding and Preventing Catastrophic Strategy Collapse in RLVR

    The authors connect abrupt performance loss to strategy concentration, derive an early-warning metric and test a strategy-balancing training method.

    MATH-500: 86.8% Mean@4 Accuracy (as reported)

    7 Oct 2026Peking University