Language models

Pretraining, architectures, post-training and instruction tuning, reasoning in language models, tokenisation, long context

339 articles · page 3 of 8

Language models

Language models can produce better writing without sacrificing variety

The authors propose PRISM, a training-time framework that internalizes output diversity in LLMs. By pairing each prompt with diverse multi-teacher solutions and injecting Gaussian noise into special-token embeddings during training, the model learns to use that noise as a signal to produce varied outputs. On writing and reasoning benchmarks, PRISM improves both quality and diversity, shifting the Pareto frontier outward.

Language models

Fine-tuning small models outperforms large ones for query clarification in RAG

The authors introduce ClariEdge, a dataset of about 3,900 examples for query clarification in retrieval-augmented generation (RAG). They show that a fine-tuned small LoRA model substantially outperforms prompting of much larger models on both classification and generation of clarifications. Hard negatives, queries that look ambiguous but are resolvable from context, are critical for performance.

Language models

Draft-training objective gains shrink when target regenerates training responses

The authors compare three draft-training objectives on external versus target-regenerated responses for a vision-language speculative decoding pipeline. Distillation and a verification gate show large gains on external responses, but after greedy regeneration, distillation adds only a small gain and the gate little or none. The shrinkage persists across controls, corpora, and a text-only replication.

Language models

LongZero trains LLMs on self-written long-context tasks, no external data

Wan and Chen propose LongZero, in which a language model synthesizes question, evidence, and answer as a compact structured specification, then expands it into a long multi-document context, and trains with GRPO. Across two base models and five long-context reasoning benchmarks, LongZero outperforms RL on synthetic data built from external corpora, generalizing to real tasks without any external data.

Language models

A language model using only self-conditioning outperforms existing diffusion models

The authors introduce LAW (Loopholing All the Way), a continuous generative model that relies purely on self-conditioning, without auxiliary conditioning mechanisms. Starting from uniformly random tokens, LAW refines the expected embedding of the clean token under the denoiser's prediction at each step and decodes to discrete tokens only after the final step. On Sudoku, Maze, and GSM8K, LAW outperforms prior discrete and continuous diffusion models, and on OpenWebText it approaches the quality of autoregressive models with 64 function evaluations.

Language models

Continuing long-context pre-training keeps improving models, and gains are predictable

Ziqing Qiao, Xiaoyue Xu, and colleagues at Tsinghua University and OpenBMB ask whether long-context continual pre-training (LCCP) yields sustained, predictable improvements as its token budget grows. They fit an additive power law to 109 training configurations across five model scales, introduce a stable checkpoint-averaging protocol for monitoring open-ended LCCP, and show that benefits persist through context extension and supervised fine-tuning without clear saturation.

Language models

Masked diffusion language models improve by encoding position priority as hierarchy levels

Gupta et al. introduce Hierarchical Masked Diffusion Language Models (HMDLM), which extend masked diffusion with a hierarchy variable per position and derive the corresponding forward/reverse processes and NELBO objective. In two-stage decoding, the model first predicts which positions are high-priority, then fills tokens conditioned on that assignment. The approach reduces perplexity by up to 7 points on WikiHow and improves Sudoku solution accuracy by up to 8.6 absolute points over diffusion baselines.

Language models

Shaping teacher behavior improves self-distillation for language models

The authors propose TeachBack, a method for on-policy self-distillation that shapes the teacher's generation behavior by controlling the length of reference reasoning and prompt style. Using a bidirectional preference objective, the teacher guides the student while the student's behavior feeds back to improve future teacher supervision. Experiments on mathematical and logical reasoning tasks show that TeachBack consistently outperforms existing OPSD methods.

Language models

Conditions for reliable reuse of intermediate results in LLM reasoning

The authors study when large language models can reliably reuse intermediate reasoning results across attempts. They derive theoretical conditions for reliable completion and bound the expected computation cost, then test eight reasoning models on composite math tasks. Providing the first subproblem's answer boosts second-subproblem success from 36.2% to 57.2%, with only a marginal further gain when full reasoning is supplied.

Language models

Prompt evolution lets frozen LLMs learn new forecasting regimes without retraining

The authors replace retraining with a continual learning loop that writes a short text guidance block into a frozen LLM's prompt. On a refreshed multimodal forecasting benchmark with strict contamination control, the evolved guidance beats three time-series foundation models on held-out data, transfers to a stronger LLM without measurable loss, and costs dollars rather than GPU-days to update.

Language models

Progressive verification speeds up diffusion language model inference

The authors propose PV-Decoding, a confidence-driven speculative decoding framework for discrete diffusion language models. It partitions masked positions into directly decoded, speculative, and unresolved tokens using two confidence thresholds, then verifies speculative tokens with progressively stronger policies. The key finding is that this approach reduces sequential model forward passes by 8–37% across three benchmarks, with the largest observed accuracy drop of 1.3%.

Language models

Gold-conditioned teacher softens targets for language model fine-tuning

The authors propose Gold-Conditioned Distillation (GCD), a self-distillation method for supervised fine-tuning of large language models. By using the same model as both teacher (conditioned on the gold response) and student (conditioned on the prompt only) and training via forward KL divergence on gold tokens, GCD achieves better accuracy on new tasks compared to standard supervised fine-tuning, while generally reducing performance degradation on general benchmarks.

Language models

New preference optimization curbs LLMs' wasteful thought switching, cuts tokens by 63%

The authors propose SteadyThought (ST), a thought-level preference optimization framework to address 'under-thinking' in large reasoning models, where models frequently switch reasoning paths even when a promising path is found. ST identifies switching points using linguistic and entropy signals, generates single-path continuations as preferred responses, and applies preference optimization to discourage unnecessary switching. The key finding: ST reduces token consumption by up to 63.8% while maintaining strong reasoning accuracy across multiple benchmarks.

Language models

Multi-hop retrieval from a shared KV cache gives hybrid models strictly deeper composition power

Eitan, Yehudai, and Maron analyze the expressive power of efficient hybrid sequence models, which pair a full-attention layer with cheaper recurrent or sparse layers. They prove that existing designs, including sparse top-κ access to the attention cache, need Ω(√n) capacity for two-function composition. Their proposed Multi-Hop Retrieval Attention (MuRA) block computes compositions of m functions at fixed size with m hops of retrieval, establishing a strict expressivity hierarchy.

Language models

Language model RL failures can guide targeted pretraining repairs

Wang and Chen show that reinforcement learning with verifiable rewards fails in recognizable, localized ways: a small set of phrases and operations strongly predicts whether a reasoning rollout will end incorrectly. They use those diagnostic n-grams to retrieve just 35 million tokens from existing training corpora, then continue pretraining before rerunning reinforcement learning, outperforming equally sized random-data controls in 10 of 11 settings.

Language models

Spherical geometry simplifies adaptive-loop reasoning and improves halting efficiency

The authors introduce Spherical Fixed-Point Reasoners (SFPRM), which perform a damped fixed-point loop on the hypersphere of normalized transformers. On Sudoku-Extreme, Maze-Hard, and Mini-ARC, SFPRM achieves test accuracies of 96.6%, 87.9%, and 59.1%, outperforming Fixed-Point Reasoners (FPRM) and other baselines while halting up to 2.7 times earlier at equal or higher accuracy. The angular geometry also provides interpretable representations: token-input angles encode puzzle structure and predict answer correctness.

Language models

Synthetic translated data improves language models across twelve Indian languages

Khattar et al. built a source-traceable synthetic pretraining corpus spanning all 22 scheduled Indian languages, with romanized text for twelve, using translation, automated quality checks and targeted repair. In controlled continued-pretraining experiments across twelve languages, models trained on a mixture of this corpus and natural Indic text outperformed a baseline based on existing public corpora, without measurably reducing English performance.

Language models

Teacher training determines whether on-policy distillation preserves prior language skills

Xu and colleagues compare supervised fine-tuning, reinforcement learning with verifiable rewards, and on-policy distillation under a shared mathematical post-training setup. They find that teacher choice strongly affects forgetting: RLVR-trained teachers can transfer mathematical gains while preserving general capabilities, whereas SFT-trained teachers tend to cause broader behavioral changes and greater capability loss.

Language models

Parallel language models match autoregressive quality while decoding several tokens per pass

The authors introduce Parallel Language Models (PLMs), which generate several mutually consistent tokens in one forward pass by routing dependencies through latent auxiliary variables. They derive a variational training objective that combines an autoregressive loss with a parallel decoding loss, and show that PLMs reach near-autoregressive pretraining quality while delivering large inference speedups.

Language models

A learned router chooses when language models should draft farther ahead

Chintha and colleagues present CADENCE, a decoding method that lets a language model choose between generating one token normally and proposing a longer continuation in parallel. On instruction following, mathematics and code benchmarks, routing drafts according to context improved acceptance relative to using the same draft length everywhere, while delivering up to 1.3 times the throughput of greedy decoding.

Language models

Switching optimizers between LLM training stages imposes no meaningful performance cost

The authors pretrained 1.2B-parameter Llama-3 models from scratch while varying the proportion of AdamW and Muon updates during pretraining, continued pretraining, and supervised fine-tuning (SFT). They found little evidence of a penalty from switching optimizers: final benchmark performance was largely explained by the independent marginal effects of each optimizer, with Muon updates during SFT generally improving outcomes regardless of the pretraining optimizer.

Language models

Coarse-to-fine sparse attention boosts long-context LLM inference speed

Prism Attention is a sparse attention mechanism for LLM decoding that reduces the overhead of selecting relevant past tokens across layers. The authors show that inter-layer token shifts are localized, and propose a two-stage method where proposal layers generate an expanded candidate pool using quantized scoring, and subsequent layers perform lightweight refinement to adapt attention to each layer. This yields state-of-the-art performance on reasoning and long-context benchmarks with a 3.4x end-to-end speedup.

Language models

Tailored reasoning traces improve student learning beyond answer-only training

The authors introduce ReDiST, a framework that constructs reasoning traces conditioned on a target student by revising its own correct or failed attempts against provided evidence. A frozen student-matched reader then selects the most teachable traces, and the student is distilled on these alongside an auxiliary answer loss. On MuSiQue and 2WikiMultiHopQA benchmarks, ReDiST outperforms answer-only, original-trace, and several baseline methods across four student models.

Language models

General-purpose LLMs already work as Jev-style decision models out of the box

Yinheng Li and Justin Wagle present LLM2Jev, a framework that converts any causal LLM into a Jev-style decision model by scoring bracketed numeric option IDs from next-token probabilities. On frozen Qwen3.5-4B, this training-free recipe matches or exceeds community Jev-style models on the same backbone and handles multimodal decisions. Fine-tuning improves weaker models and many-option routing, but yields diminishing returns for strong backbones.