Language models

Pretraining, architectures, post-training and instruction tuning, reasoning in language models, tokenisation, long context

339 articles · page 5 of 8

Language models

Loop-native residuals let Transformers reason across tens of thousands of iterations

The authors introduce InfiLoop, a residual connection for looped Transformers that learns to selectively aggregate past recurrent states instead of overwriting them with each new iteration. On Sudoku-Extreme, a 7M-parameter InfiLoop model reaches 97.9% exact accuracy, and it continues to improve even beyond 20,000 effective steps, demonstrating that added loop depth can translate directly into stronger reasoning.

·2 min
Language models

A pipeline that generates diverse multimodal instruction data from simple requirements

The authors propose UniData, a universal pipeline that transforms simple user requirements into multi-round, multimodal instruction datasets. They also introduce UniDataset, a dataset of 20,000 entries spanning nine modalities. Experiments show that UniData achieves state-of-the-art data quality in reasonableness, clarity, detail, and relevance, and that the generated data can enhance the understanding and generation capabilities of other multimodal large language models.

·2 min
Language models

Offline student rollouts often beat on-policy distillation for language models

Zhao et al. propose Semi-OPD, a distillation method that reuses a single set of offline student rollouts sampled before training rather than regenerating them on-policy at each step. They show that Semi-OPD outperforms standard on-policy distillation (OPD) on most teacher-student pairs, especially when the teacher and student are poorly aligned. The paper provides a practical metric, the token overlap ratio, to decide which method to use.

·3 min
Language models

Bi-encoders can support multimodal decisions across small and massive choice sets

Cheng et al. turn a pretrained 30-billion-parameter Muse-Glimmer decoder into a multimodal bi-encoder that matches requests to independently encoded candidates. Across 11 benchmark suites and 190 tasks, the authors report stronger results than existing multimodal embedding models on every evaluated decision-making task, near-leading retrieval performance, and persistent weaknesses on problems requiring complex reasoning.

·3 min
Language models

Morphology-aware tokenizer beats compression-only baselines across 104 languages

Ali et al. introduce the Latent Core Tokenizer (LCT), which first learns reusable linguistic units using minimum description length, branching entropy, and morphotactic constraints, then builds a surface vocabulary by recombining those units. Across 104 languages with a 200K token budget, LCT achieves lower fertility and higher MorphScore than BPE, Unigram, and parity-aware BPE, and improves aggregate downstream scores by 1.48 to 2.00 points in controlled encoder pretraining.

·2 min
Language models

Prior barriers quantify uneven pretrained concept support for fine-tuning

The authors introduce 'prior barriers' to measure how strongly a pretrained model favors competing concepts over a target concept. They observe these barriers follow a long-tail distribution, meaning frequent concepts need little additional evidence while rare concepts demand more instructions. They propose PASS, an adaptive instruction selection method that allocates budget to under-represented concepts, and show it outperforms seven baselines across four backbone-budget settings.

·2 min
Language models

Supervision reliability trumps alignment coverage in cross-tokenizer distillation

The authors investigate cross-tokenizer on-policy distillation (OPD) for language models. Across three heterogeneous teacher-student pairs on math and code tasks, they find that strict 1:1 token alignment covers 85–97% of tokens and retains nearly all probability mass in the shared vocabulary. A compact student-selected top-16 vocabulary subset matches full shared-vocabulary OPD, losing less than 4% of the improvement. Adding mean squared error (MSE) supervision on mismatched spans—which achieves complete alignment coverage—consistently worsens accuracy; gradient diagnostics show that span gradients conflict with strict gradients and grow in relative magnitude.

·3 min
Language models

Separate short- and long-term matrix memories improve recurrent network performance

Regan et al. introduce the Continuous Memory Machine (CMM), a recurrent architecture with two matrix-valued memory states for short- and long-term storage, updated jointly by a shared Transformer. The CMM outperforms a suite of baselines across algorithmic, in-context learning, and maze-solving tasks, and shows better generalization than prior memory-augmented networks while preserving interpretable attention patterns.

·2 min
Language models

Controlling gradient admission stabilizes LoRA fine-tuning of small language models

The authors propose GRADE, a data-centric framework that controls which data-induced gradients enter the LoRA subspace during SLM fine-tuning. Across three backbones and seven heterogeneous instruction datasets, GRADE consistently improves accuracy over standard LoRA and outperforms existing selection and stabilization methods, demonstrating that successful adaptation depends on managing gradient admission under constrained update capacity.

·3 min
Language models

Bayesian memory module adapts reasoning strategies during test time

The authors introduce the Bayesian Cheatsheet, a structured memory module for large language models that uses a Hierarchical Dirichlet Process Gaussian Mixture Model (HDP-GMM) to organize reasoning behaviors into an unbounded number of clusters. In online test-time training, the module retrieves relevant clusters for a query, updates cluster statistics after each sample, and can create new clusters for novel behaviors. On reasoning benchmarks like AIME'25, Omni-MATH, and PhysReason, Bayesian Cheatsheet outperforms existing memory modules, even in cold-start settings.

·3 min
Language models

Difficulty-gated teacher guidance improves solution coverage in on-policy training

Tiwari, Chordia and A.P. combine on-policy distillation with Group Relative Policy Optimization (GRPO), a reinforcement-learning method that compares rewards among multiple sampled solutions. Their DiffGate objective uses correctness feedback to target teacher guidance at failed solutions, especially when every rollout fails, improving the chance that Qwen3 students produce at least one correct answer across eight attempts.

·2 min
Language models

Adaptive token computation narrows diffusion language models’ quality gap

Fang et al. modify diffusion language models so that different masked tokens can receive different amounts of computation during each denoising step. At 1.7B and 8B parameters, their models achieved higher average scores across eleven reasoning, mathematics and coding benchmarks than all evaluated diffusion models and corresponding autoregressive baselines, while the 8B model retained a reported speed advantage on GSM8K.

·2 min
Language models

Human preferences contain signals that rubrics and verifiers miss

Spangher et al. analyze 317 million human preference judgments over 2.8 million texts to measure how much evaluative judgment can be verified, articulated in rubrics, or learned only from examples. They find persistent articulability gaps in all seven domains, including mathematics and software engineering, and show that emphasizing explicit criteria can reduce alignment with otherwise unexplained human preferences.

·3 min
Language models

Personalized LLMs can share most adaptation capacity across users, needing only tiny per-user code.

The authors revisit personalized adaptation for large language models by asking how much of a personalized adapter is actually personal. Through empirical analyses, they find that independent user adapters contain substantial reusable structure and that user history provides transferable correction signals. They propose LINEUP, which learns a bank of reusable low-rank personalization factors, composes them via user-conditioned recall and query-dependent calibration, and restricts target-user adaptation to a tiny eight-scalar code. Across six LaMP tasks spanning classification, prediction, and generation, LINEUP achieves the best mean performance on all 12 reported metrics, demonstrating that rich personalization can be supported primarily by shared capacity.

·3 min
Language models

Reinforcement learning with verifiable reward causes unbounded language drift in LLM reasoning chains

The authors investigate language drift in LLM chains of thought during reinforcement learning with verifiable reward (RLVR) post-training. They prove that RLVR permits unbounded drift, unlike supervised fine-tuning which has a finite upper bound, and that limiting drift limits reward. Empirically, drift arises specifically on novel reasoning tasks where the target behavior cannot be drawn out of the base model, and different RLVR runs develop unique drifts.

·2 min
Language models

Bayesian fine-tuning instills usable beliefs in language models but stops short of genuine Bayesian reasoning

The authors compare two fine-tuned language models on a flight recommendation task: one trained on outputs of a Bayesian assistant (BayesLM) and one on true preference labels (OracleLM). BayesLM acts as a Bayesian decision maker, represents the required beliefs in its middle layers, and uses them causally to make choices. OracleLM matches behavior but diverges in its internal representations and how it reads beliefs into actions, showing that standard supervised fine-tuning does not install usable Bayesian beliefs.

·3 min
Language models

Spectral theory diagnoses two failure modes in RoPE long-context models

The authors extend theoretical analysis of Rotary Position Embedding (RoPE) to unequal query-key scales, revealing two fundamental failure modes: semantic reversal (position changes reversing token preferences) and positional insensitivity (adjacent tokens becoming indistinguishable). They introduce RoPE Profiler, a lightweight diagnostic that computes Semantic and Positional Scores by reusing cached activations from standard evaluation, and show that targeted high-frequency rescaling guided by these diagnostics improves accuracy by up to 20 percentage points on Qwen3-8B and 25 points on Llama-3.1-8B-Instruct across 49 long-context task settings.

·3 min
Language models

One LLM mechanism handles both retrieval and long-context evidence selection

Kinderman et al. present UNREAL, a framework that turns evidence retrieval and long-context filtering into the same model-internal operation. The system adds fewer than 500,000 trainable parameters to a frozen decoder-only language model, using its hidden states both to encode candidate chunks and derive retrieval queries. Across the reported benchmarks, it improves corpus retrieval and removes distracting text before generation, reducing computation for sufficiently long inputs.

·3 min
Language models

Role prompts help language models only under the right conditions

Zhuang et al. test whether asking a language model to act as an expert reliably improves knowledge-based question answering. On MMLU and MMLU-Redux, they find that role prompts can help, do nothing or reduce accuracy depending on the model, domain and prompt language. Their training-free MLCP method, which concatenates equivalent role descriptions in multiple languages, outperforms conventional single-language role prompting across every tested model.

·3 min
Language models

Sequential test-time scaling can surpass parallel scaling by escaping attractors

Rance et al. show that sequential test-time scaling for language model reasoning often stalls because models get trapped in 'attractors': sets of answers that block exploration. Across 27 combinations of methods, models, and benchmarks, 53.8% of trajectories hit an attractor within four iterations. A simple intervention—switching the underlying model during scaling—reduces the attractor rate by 21.2 percentage points on average and lets sequential methods outperform compute-matched parallel baselines.

·2 min
Language models

Routing attention heads by token enables joint scaling of parameters and context

Fu et al. introduce NAMOH, an architecture-native sparse attention mechanism that treats each attention head as an expert. A learned router selects K out of H heads for each token, and each head performs causal attention only on the tokens routed to it. This design reduces per-token KV access to approximately (K/H)² relative to full H-head attention, while experiments show NAMOH outperforms dense models with the same total parameters and enables more efficient long-context inference than smaller dense models matched in active parameter count.

·3 min
Language models

Byte Transformers outperform subword models at scale and learn implicit token-like abstractions

Jie Wang, Shiwei Luo, Qi Zhang, and Yuanbin Wu train flat byte Transformers without specialized tokenization architectures and show that they outperform subword Transformers as model size scales. The models also learn local text abstractions similar to tokenizers, with segmentation positions that can be exploited for speculative decoding, yielding 3.4 times more accepted tokens.

·2 min
Language models

Activation steering strengthens token-level supervision for self-distilled reasoning

Lu et al. introduce Activation-Conditioned Self-Distillation (ACSD), which turns differences in a model’s own reasoning trajectories into dense next-token training signals. Across five language models, ACSD achieved the highest average accuracy among the evaluated methods on four mathematics benchmarks. It also improved code-generation results on the tested 8-billion-parameter model, while requiring no reference solutions during distillation or activation intervention at inference.

·2 min
Language models

Starting token cues can elicit reasoning from base language models

The authors demonstrate that base language models can perform complex reasoning if their output begins with specific short token sequences (e.g., "Okay" or "Alright,"). These cues, learned from training data, boost MATH-500 accuracy from 42% to 78% for Olmo-3-7B, matching its RL-trained counterpart. The work further shows that RL essentially increases the likelihood of such cues, and that causal data interventions can turn arbitrary words into effective reasoning triggers.

·3 min
Language models

Tree-based routing balances expert use without auxiliary loss in MoE models

The authors propose BRANCH-MoE, a routing architecture that places experts at leaves of a binary decision tree. By centering branching probabilities on arrival-weighted mean scores estimated via exponential moving average, the method prevents routing collapse and provides theoretical convergence guarantees. Experiments show it matches or outperforms flat routers on four tabular benchmarks while inducing localized expert co-activation.

·3 min
Language models

Blending teacher and student token distributions improves distillation of language models.

Shi et al. propose Interpolated Policy Distillation (IPD), which defines the rollout policy at each decoding step as a linear interpolation between student and teacher token distributions, governed by a single coefficient. They accelerate exact sampling with a speculative decoding rule that preserves the interpolated distribution. IPD consistently outperforms off-policy SFT, on-policy distillation, their two-stage combination, and recent segment-interleaving baselines on text-only and multimodal reasoning benchmarks.

·3 min
Language models

Averaging checkpoints across shorter training runs beats one long mid-training run

The authors propose Trajectory Soup, a strategy that forks multiple independent branches from a shared pretrained checkpoint, averages the strongest checkpoints within each branch, and then averages the resulting anchors across branches. Under matched compute budgets, the merged model outperforms the strongest single-trajectory average and keeps improving as more trajectories are added, even after a standard post-training pipeline.

·2 min
Language models

On-policy distillation can transfer think-mode advantage to no-think responses via trace-response routing

The authors identify trace-response divergence (TRD) in a natural distillation setting where a think-mode teacher provides privileged supervision for a no-think student. They propose ThinkOPD, which combines reward gain and TRD to weight student responses within each group, improving average performance by 1.7–3.0 percentage points over uniform weighting across same-model experiments.

·2 min
Language models

Low-rank router enables scaling mixture of experts to many more experts

The authors propose MoRE, which replaces the standard linear router in mixture-of-experts layers with a low-rank factorization, reducing per-token cost from Θ(Mh) to O((h+M)r). They prove that rank logarithmic in the number of experts suffices for routing expressivity and load balancing, and demonstrate empirically that MoRE improves memorization and knowledge-intensive QA while matching reasoning ability at equal active FLOPs, with a fused kernel for efficient inference.

·3 min
Language models

Tunable attention masks enable subquadratic long-context sequence modeling

The authors introduce Structured Matrix Attention (SMat-Attention), a family of causal attention masks parametrized by VC dimension d. For d ≤ 3, prefill is O(T); for d > 3, it is O(T^{2-3/d}). After a fixed prefix, decoding takes constant time per token using O(T^{1-1/d}) cached states. The approach is extended to Mamba-2 and Gated DeltaNet, improving recall accuracy while remaining competitive in language modeling.

·2 min
Language models

Activation-guided subspaces improve forward-only fine-tuning of large language models

Xie and colleagues introduce AIM-ZO, a zeroth-order fine-tuning method that estimates model updates using only forward loss evaluations. Across five language models and 11 tasks, it generally outperformed prior forward-only methods under matched evaluation budgets, including gains on models with 2.7 billion and 30 billion parameters. The results suggest that carefully maintaining perturbation directions can make forward-only optimization more effective without sacrificing its low-memory premise.

·2 min
Language models

Two-sided associative memory correction boosts linear attention performance

The authors introduce Gated Slot Attention-2 (GSA2), which combines Gated Oja Rule-2 for key-side correction and Gated Delta Rule-2 for value-side correction via shared latent slots. At 1.3B scale, GSA2 consistently outperforms strong linear-attention baselines on language modeling, commonsense reasoning, synthetic retrieval, and long-context tasks, with particularly clear gains on RULER and LongBench.

·2 min