News without the noise
Pretraining, architectures, post-training and instruction tuning, reasoning in language models, tokenisation, long context
66 articles · page 1 of 2
Zhang et al. propose a framework for multi-task models that learns explicit cross-task relationships by approximating the joint distribution of task labels via pairwise interactions. Tested on YouTube's Notifications, Homepage, and Watch Next surfaces, the method improved both accuracy and user satisfaction metrics without the complexity of full joint modeling.
Yu and colleagues test whether an on-policy context-distillation teacher should receive each problem’s gold answer or a short instruction aimed at common student errors. Across three logic datasets and two language-model families, matched instructions retained in-domain performance while improving out-of-distribution accuracy by 4 to 17 percentage points in seven of eight comparisons.
Gahtan et al. introduce effective depth, a scalar that treats a transformer's residual stream as a discrete-time process and measures how representation similarity decays with layer distance. Across 16 decoder-only LLMs, they find that effective depth is almost always below a closed-form reference that assumes maximally diverse orthogonal updates, suggesting that residual updates are strongly correlated.
Agarwal et al. introduce BOReFT, a method that turns hidden-state interventions in a frozen language model into a compact continuous domain for black-box optimization. Across a semantic word-search benchmark and three molecular-design objectives, the method improved target discovery in Semantle and outperformed strong language-model baselines on two of the three molecular objectives.
The authors propose Teacher Alignment to address the Gap Curse in reasoning distillation, where larger teacher models produce distributions too complex for smaller students to approximate. They develop TeacherGRPO, built on Group Relative Policy Optimization, which adapts the teacher via curriculum selective alignment and importance-adaptive length regularization. Experiments show consistent performance gains over existing baselines across diverse reasoning benchmarks.
The authors developed a behavioral test suite to distinguish whether language models rely on shallow patterns or true generalization. Tracking models across pre-training checkpoints, they found that models frequently and abruptly alternate between these modes, often within tens of billions of tokens. This mode-hopping is not an artifact of noise or metric selection and cannot be smoothed by checkpoint averaging.
Liang and colleagues introduce Allspark, a framework for transferring reasoning behavior from a reinforcement-learned small model to stronger models through alternating text-based chains of thought. In experiments using Qwen models and ARC-AGI-2, the trained weak teacher improved some within-family and cross-family students, although the benefit depended on the inference configuration and required additional tokens.
Zhou et al. test whether relationships learned from persona prompts predict new responses and carry across models and prompts. They find persona effects are predictable within a model but not portable: borrowed gains underperform target refitting in most transfers, and rewording consistently breaks reuse.
Guo and colleagues trained a 200 million-parameter Transformer whose computation can stop after any layer, yielding a continuum of smaller language models from one artifact. Unlike suites trained only at selected exit depths, their Telescopic Language Model remained functional at all 20 prefixes and improved the reported quality-budget tradeoff without changing the architecture or adding inference overhead.
Ding, Zazo and Hensman introduce Rotated Manifold Optimization, or RoM, an optimizer designed around a symmetry in low-rank adaptation: many pairs of LoRA factors represent exactly the same weight update. Across supervised fine-tuning and reinforcement learning experiments, RoM converged faster and reached lower held-out loss than the compared LoRA optimizers, while producing better or comparable downstream performance.
Zeng and colleagues examine whether every token in a chain-of-thought trace contributes equally to solving a problem. They report that normalized token log probabilities can identify compact regions containing decisive reasoning, then use this signal in reinforcement learning to reduce token consumption by 76% relative to the baseline without sacrificing overall reasoning quality.
Yin and colleagues tested a local learning method that lets network modules update independently rather than waiting for gradients to traverse the entire model. Across Transformers pretrained on 15 billion tokens, SOLO remained within one percentage point of backpropagation in average zero-shot accuracy and achieved up to 1.44 times its best measured pipeline throughput.
The authors introduce d-OPD, a future-aware on-policy distillation method for converting autoregressive LLMs into block diffusion language models. It addresses a fundamental mismatch: the block-diffusion student sees future context within a block, but the standard AR teacher distribution does not. By aligning the teacher distribution with the student's visible state, d-OPD improves average benchmark scores by up to 4.0 points and reduces training time by about 1.35-1.58x on Qwen3 models from 0.6B to 8B.
Zhaohan Zhang and colleagues propose ConfLens, a framework that monitors how an LLM's confidence in its final answer evolves during chain-of-thought reasoning. They find that shortcut reasoning consistently shows a pattern of premature confidence, where the model commits to an answer early. Their new metric, the Distributional Answer Commitment Score (DACS), detects this reliably across math and code tasks, improving shortcut detection F1 by over 4.3% over strong baselines.
Chen and colleagues compare multimodal language models that use a pretrained visual encoder with models that instead feed projected image patches directly into the language decoder. Encoder-free models are less efficient at the tested scales, but their multimodal loss improves faster with compute, leading the fitted scaling laws to predict parity around 10^22 FLOPs.
Lévy and colleagues identify slow rotary frequency bands, whose wavelengths exceed the training context, as a weakness of Rotary Position Embedding during length extrapolation. Their alternative, Data-aware RoPE, performs best or comparably to other tested encodings across language models, synthetic tasks, music, genomics and neural signals, while reducing the tendency to favor recent tokens.
Qiu and colleagues modify mixture-of-experts routing so each layer can consult the expert-selection distributions produced for the same token by preceding layers. In an approximately 8 billion parameter model trained from scratch on 100 billion tokens, the method reduced final training loss from 1.6393 to 1.6184 while adding modest computational and memory costs.
Ma et al. introduce Leverage Score Sampling (Lev), an iterative selection method that uses leverage scores to maximize data diversity efficiently, eliminating costly matrix recomputation. On CommonCrawl data, Lev improves accuracy by up to 1.31% over strong baselines and reduces bits-per-byte on code data by 3.08% relative to DiSF.
The authors develop in-situ representation refinement, where support labels guide changes to the episode's representations during a forward pass. They propose RefineICL, an attention-gated, FFN-free contextual stack that achieves 0.93836 OVR-AUC and 0.87173 accuracy on AMLB29 and 1644.8 Elo on TabArena, 31.4 above TabPFN-3. A direct intervention confirms that evolving support states are essential: removing one intermediate support update increases final query cross-entropy in all 72 tested episodes.
Tallini et al. introduce LocUS, which uses the unembedding matrix to define a property-specific subspace and selects a sparse subset of attention heads. Steering is applied only within that subspace and on those heads. The method matches or outperforms standard steering baselines on three tasks while intervening on under 6% of parameters and better preserving general capabilities.
The authors systematically evaluate whether token-level certainty scores from LLMs predict correctness, distinguishing between identifying questions a model will likely answer correctly and distinguishing correct from incorrect responses to the same question. They find that certainty is generally better at the former, and that the timing of useful information differs: question difficulty appears early, answer correctness appears late. A test-time compute strategy built on these findings raises accuracy from 78.71% to 79.54% while cutting generated-token cost by 82.4%.
The authors propose APIVIS, a training-time framework that integrates finite-budget Gumbel search into reinforcement learning with verifiable rewards (RLVR) for mathematical reasoning. By combining direct and searched responses in each rollout group and applying selective supervision to search-improved tokens, they achieve substantial gains over existing search-based methods across benchmarks and model scales.
The authors propose RPTune, a framework that learns how to curate product catalogs for long-context LLMs and fine-tunes the LLM to better use the curated input. Evaluated on 7 real merchants, context curation alone improved search accuracy by up to 31.4 percentage points, and post-training added a further 10.3 points on average.
Geng et al. present an inference-time decoding method that separates high-level idea selection from token-level generation. Across chat, creative writing, image-generation planning and a protein-design pilot, the authors report substantially greater diversity at comparable quality than existing sampling and prompting methods.
The authors formalize multi-path LLM reasoning as a diversity combining problem. They show that pairwise correctness correlation among reasoning paths limits the effective sample size of majority vote, causing saturation. Using an Adaptive-K rule, they can select an optimal number of paths, retaining 96-103% accuracy of K=32 with fewer paths.
The authors introduce On-Policy Attention Self-Distillation (OPASD), which adds solution-conditioned attention alignment to token-level on-policy self-distillation. Across three model sizes and four competition-level math benchmarks, OPASD consistently outperforms token-only distillation, improving average accuracy by 4.98 to 8.40 percentage points and reducing generated tokens by 73.9%.
Tirosh, Amos and Geva introduce LIFT, a Transformer architecture that feeds a compact continuous state from each generated position into the processing of the next. Across models from 135 million to 1 billion parameters, the authors report better language modeling, reasoning and procedural-task performance than standard Transformers trained on the same number of tokens, with competitive or better results when training compute is matched.
Shao and colleagues introduce Context Language Models, or CLMs, which let a language model directly rewrite the information it retains rather than relying on an external harness to summarize or discard context. Across several long-horizon benchmarks, existing models using this design outperformed engineered context-management strategies while consuming fewer floating-point operations, or FLOPs.
The authors introduce RECAP, a redundancy-aware credit assignment method that assigns step-level rewards based on both a step's downstream role in the reasoning structure and its progress toward the correct answer. Applied to two 7B models across four math benchmarks, RECAP improves pass@1 while cutting reasoning tokens by 8%-31% relative to GRPO.
Pan et al. introduce a model for continuous interaction with streaming time-series data, rather than waiting for a complete sequence or pausing input while responding. On their StreamTSI-34K benchmark, TimeInteract outperformed language, vision-language, and time-series language models while nearly eliminating stream stalls.
The authors propose concurrent confidence calibration, where confidence is learned alongside capability improvement during reinforcement learning from verifiable rewards (RLVR). Their method, CoCal (Companion Confidence Calibration), trains a separate lightweight model on rollout hidden states and verifier feedback, decoupling confidence estimation from policy optimization. Experiments on Qwen3-8B and Qwen3-14B show that CoCal improves confidence estimation without sacrificing task accuracy, outperforming both RL-based concurrent methods and post-hoc calibration.
The authors propose a method where a small concept generator is trained using reinforcement learning to produce high-level concepts (e.g., hints, strategies) that maximize the downstream success of a frozen, larger answer generator. On hard mathematical reasoning problems, this learned search policy substantially improves pass@k over naive repeated sampling at the same answer generation allocation, transfers to other answer generators, and outperforms concepts from much larger untuned models.
The authors introduce a form of on-policy distillation designed for prompts containing many simultaneous constraints. Across two Qwen teacher-student pairs and seven benchmarks, their method achieved the highest average performance among the evaluated training approaches, and a 1.5B student surpassed its 7B teacher on MulDimIF.
Wu et al. replace independently sampled reasoning attempts with a planner that jointly assigns different solution outlines before a fixed model executes each one. Across five mathematical reasoning benchmarks, their zero-shot method improved pass@64 by up to 6.7 percentage points over repeated sampling, while a reinforcement-learned planner improved it by up to 13.4 points.
Zhao, Yu and Zhao study stable-but-wrong lock-in, where a diffusion language model settles early on an incorrect answer despite having denoising steps left. Their LOCKR planner uses the trajectory of internal hidden states to trigger additional computation and choose repairs, improving mathematical reasoning accuracy by 2.21 to 5.37 percentage points across five settings.
The authors propose MOPD-Router, a framework for multi-teacher on-policy distillation that replaces prompt-level hard routing with token-level routing across all teachers, requiring no domain labels and no separate routing model. Within this framework they introduce ExpertAlign, a metric that scores each teacher by whether its correction to the student expresses the specialization the teacher acquired during post-training. On unlabeled data, ExpertAlign improves overall scores by 12.3% over Mean aggregation; on domain-labeled data, it beats standard MOPD by 7.8% without using the available domain labels.
Althoubi compared reasoning and non-reasoning modes across five benchmarks using 74,944 samples, isolating the effect of reasoning while holding model weights fixed. Reasoning made independently sampled errors more likely to coincide, while confidence-weighted voting failed to improve reliably over a simple majority.
Zhang, Wu and Li compare supervised fine-tuning datasets selected from the same candidate pools under matched training budgets, differing primarily in the diversity of their reasoning routes. Models trained on diverse routes generally covered more problems after reinforcement learning, including environments absent from supervised training.
Chen et al. separate long-context processing into localized evidence extraction and centralized reasoning, rather than asking one model to search and reason over the entire input. Their system achieves 78.4% accuracy on 1-million-token RULER-QA and reports stronger LongBench v2 performance at substantially lower inference cost than full-context baselines.
Li and colleagues introduce Autoregressive Thought Flow, which represents each next latent thought as a multimodal distribution rather than a single predicted state. On mathematical reasoning tasks, the method improves accuracy and sampled solution coverage, while benefiting from reinforcement learning and additional test-time reasoning steps.
The authors propose SAGE, a training-free inference-time framework that structures LLM strategic reasoning around three coordinated operations: anchor, adapt, and recalibrate. In repeated imperfect-information games (Leduc Hold'em, Liar's Dice, Goofspiel), SAGE outperforms reasoning-intensive LLM agents, achieving up to a 127.6% payoff improvement in Liar's Dice while reducing input and output token usage by up to 80% and 90% respectively.
Bu and colleagues isolate the cost of information loss at interfaces in four-stage LLM pipelines, which they call the decomposition tax. Across 21 open-weight models, re-grounding later stages with the original problem recovered substantial accuracy, with the largest measured tax reaching 40.5 percentage points.
Huang and colleagues compared base and aligned language models on long-tail factual questions, then traced how confidence in incorrect answers developed across model layers. They report that instruction-tuned models produced 10 to 35 times more errors with probabilities of at least 0.95, and that a modified alignment objective reduced these errors by up to 35.3 percent.
Yin et al. modify on-policy self-distillation so that a solution-informed teacher periodically incorporates improvements learned by its student. Combined with training on shorter, verified rewrites, the method raised Qwen3-8B's reported Average@12 score to 65.97 percent, 35.62 percentage points above standard on-policy self-distillation.
Chang et al. post-trained GLM-4.5-Air-Base, a mixture-of-experts model with 106 billion total parameters and 12 billion active per token, using public data and open-source components. They report that the resulting Rufus-Air model outperforms the official post-trained GLM-4.5-Air release and is competitive with similarly sized open models.
Hosseini et al. trained reasoning models to estimate confidence at intermediate points in their own reasoning, using labels derived from the models' token probabilities and only 600 training problems. The resulting models generated up to 25% fewer tokens at matched accuracy, despite receiving no objective to reason briefly or stop early.
Xia et al. train a model to handle long inputs in three explicit stages: identify question-relevant evidence, combine it into a compact summary, then answer. On a seven-task evaluation suite, H2S-14B achieved an average score of 32.60 and retained 97.1% of its performance when its output budget was reduced from 16K to 4K tokens.
Zeng et al. analyze why outcome-based training becomes brittle when successful reasoning paths are long and rewards are rare. Their SAGE framework adds structural priors to guide exploration, outperforming reported baselines across 12 benchmarks and seven model families, with up to an eightfold improvement on the Andrews-Curtis task.