Language models

Pretraining, architectures, post-training and instruction tuning, reasoning in language models, tokenisation, long context

339 articles · page 7 of 8

Language models

Tabular foundation models refine representations in-context with attention-gated updates

The authors develop in-situ representation refinement, where support labels guide changes to the episode's representations during a forward pass. They propose RefineICL, an attention-gated, FFN-free contextual stack that achieves 0.93836 OVR-AUC and 0.87173 accuracy on AMLB29 and 1644.8 Elo on TabArena, 31.4 above TabPFN-3. A direct intervention confirms that evolving support states are essential: removing one intermediate support update increases final query cross-entropy in all 72 tested episodes.

·2 min
Language models

Token-level certainty predicts LLM question difficulty better than response correctness

The authors systematically evaluate whether token-level certainty scores from LLMs predict correctness, distinguishing between identifying questions a model will likely answer correctly and distinguishing correct from incorrect responses to the same question. They find that certainty is generally better at the former, and that the timing of useful information differs: question difficulty appears early, answer correctness appears late. A test-time compute strategy built on these findings raises accuracy from 78.71% to 79.54% while cutting generated-token cost by 82.4%.

·3 min
Language models

Value-guided search improves math reasoning during RL training

The authors propose APIVIS, a training-time framework that integrates finite-budget Gumbel search into reinforcement learning with verifiable rewards (RLVR) for mathematical reasoning. By combining direct and searched responses in each rollout group and applying selective supervision to search-improved tokens, they achieve substantial gains over existing search-based methods across benchmarks and model scales.

·2 min
Language models

Latent feedback helps Transformers carry information across generation steps

Tirosh, Amos and Geva introduce LIFT, a Transformer architecture that feeds a compact continuous state from each generated position into the processing of the next. Across models from 135 million to 1 billion parameters, the authors report better language modeling, reasoning and procedural-task performance than standard Transformers trained on the same number of tokens, with competitive or better results when training compute is matched.

·3 min
Language models

Companion model sharpens LLM confidence without harming task performance

The authors propose concurrent confidence calibration, where confidence is learned alongside capability improvement during reinforcement learning from verifiable rewards (RLVR). Their method, CoCal (Companion Confidence Calibration), trains a separate lightweight model on rollout hidden states and verifier feedback, decoupling confidence estimation from policy optimization. Experiments on Qwen3-8B and Qwen3-14B show that CoCal improves confidence estimation without sacrificing task accuracy, outperforming both RL-based concurrent methods and post-hoc calibration.

·3 min
Language models

Training a concept generator improves LLM reasoning over repeated sampling.

The authors propose a method where a small concept generator is trained using reinforcement learning to produce high-level concepts (e.g., hints, strategies) that maximize the downstream success of a frozen, larger answer generator. On hard mathematical reasoning problems, this learned search policy substantially improves pass@k over naive repeated sampling at the same answer generation allocation, transfers to other answer generators, and outperforms concepts from much larger untuned models.

·2 min
Language models

Token-level teacher routing beats prompt-level routing in multi-teacher distillation

The authors propose MOPD-Router, a framework for multi-teacher on-policy distillation that replaces prompt-level hard routing with token-level routing across all teachers, requiring no domain labels and no separate routing model. Within this framework they introduce ExpertAlign, a metric that scores each teacher by whether its correction to the student expresses the specialization the teacher acquired during post-training. On unlabeled data, ExpertAlign improves overall scores by 12.3% over Mean aggregation; on domain-labeled data, it beats standard MOPD by 7.8% without using the available domain labels.

·2 min
Language models

Structured reasoning framework improves LLM game-playing efficiency and payoff

The authors propose SAGE, a training-free inference-time framework that structures LLM strategic reasoning around three coordinated operations: anchor, adapt, and recalibrate. In repeated imperfect-information games (Leduc Hold'em, Liar's Dice, Goofspiel), SAGE outperforms reasoning-intensive LLM agents, achieving up to a 127.6% payoff improvement in Liar's Dice while reducing input and output token usage by up to 80% and 90% respectively.

·3 min
Language models

Hybrid backbones can efficiently adapt into diffusion language models

The authors adapted pretrained Qwen3.5 models at four parameter scales into diffusion language models, which generate by iteratively filling masked positions rather than strictly proceeding left to right. Despite the backbone’s structurally causal recurrent layers, the resulting dQwen3.5 models retained any-order and parallel decoding behavior while adapting more efficiently than a full-attention baseline.

·2 min
Language models

Sparse mixture-of-experts models overfit repeated training data faster

Jha et al. trained dense Transformers and mixture-of-experts models under varying data-repetition rates, domain mixtures, expert counts, and expert granularities. They find that MoE models lose performance more rapidly than dense models as examples are reused, with susceptibility tracking total parameter count rather than the number of parameters activated per token. Strong masking-based regularization preserves an MoE advantage beyond 64 repetitions, but does not match training on unique data.

·2 min
Language models

Hierarchical memory trees improve long-context reasoning without model training

Zhang et al. introduce ConvMem, a training-free framework that uses prompted language models as query-dependent summarization kernels over segments of long documents. On two synthetic long-context, multi-hop question-answering benchmarks, the authors report better performance than training-free baselines and stronger out-of-distribution behavior than reinforcement-learning-trained memory methods.

·2 min
Language models

Distillation Before Reinforcement Learning Improves Reasoning Model Post-Training

Li and colleagues compare ways to combine on-policy distillation (OPD), which supplies dense teacher feedback at the token level, with reinforcement learning from verifiable rewards (RLVR), which uses sparse outcome-based signals. Across logic and mathematics benchmarks, they find that completing OPD before switching to RLVR works consistently better than pure OPD, pure RLVR, or joint optimization.

·2 min
Language models

Graph Machine architecture uses dynamic sparse routing to handle large state efficiently

The authors introduce the Graph Machine (GM), an architecture that maintains an O(n)-sized state and accesses it via sparse, dynamic routing using edges—pointer-like objects updated by a referral mechanism. They replace 75% of the dense Transformer layers in Qwen3-0.6B with GM sparse layers and pretrain from scratch on 15.7B tokens. With only 2 of 4,096 tokens retrieved per KV head per sparse layer, loss degrades only slightly; with 4 tokens, the best model marginally improves loss over the dense baseline.

·2 min
Language models

Self-evolving loop synthesizes high-quality multimodal training data

The authors present VISA, an agentic framework that reformulates multimodal instruction synthesis as a self-evolving loop. VISA iteratively generates diverse and challenging instruction-following samples, using verifier signals and target-model failure profiles to guide subsequent rounds. The method consistently improves multimodal instruction following on MM-IFEval while maintaining general multimodal capability across seven benchmarks.

·2 min
Language models

Sliding reasoning windows make long test-time scaling 3x faster

Muennighoff et al. observe that during long reasoning traces, most intermediate tokens lose importance as the model continues. They propose Prefix Sliding, which keeps only the instruction prefix and a recent window of tokens, discarding the rest to cap memory usage. Applied without training to existing models, it achieves up to 3x speedup with no performance loss; training with it enables reasoning traces beyond 100,000 tokens.

·2 min