Language models

Pretraining, architectures, post-training and instruction tuning, reasoning in language models, tokenisation, long context

66 articles

All Agents Language models Reinforcement learning Vision Generative media Speech & audio Robotics & embodied AI Safety & security Evaluation & benchmarks Efficiency & systems Code & software AI for science & health Theory

Viewgrid list compact

Modeling pairwise task relationships improves multi-task recommendation systems

Short error-targeted instructions improve distillation across logic datasets

Effective depth shows transformer layers are redundantly correlated

Low-dimensional language-model steering supports more systematic black-box search

Teacher alignment method closes capacity gap in reasoning distillation without discarding data

Language models can suddenly and repeatedly switch between pattern-matching and genuine generalization during pre-training

Small models can steer stronger ones through shared reasoning traces

Persona effects in language models are predictable but not portable

One nested language model works across every tested layer depth

Geometry-aware LoRA optimization improves convergence across supervised and reinforcement learning

Language models can cut reasoning tokens by targeting low-value text

Shared readouts bring local learning close to backpropagation at billion scale

Future-aware distillation corrects teacher-student information mismatch in block diffusion language models

Early answer confidence reveals when language models take reasoning shortcuts

Encoder-free multimodal models could close the gap with enough compute

Data-aware rotary encoding reduces recency bias across diverse sequence tasks

Reusing Earlier Expert Choices Improves Sparse Model Training Efficiency

Leverage score sampling offers scalable diversity for language model pretraining data selection

Tabular foundation models refine representations in-context with attention-gated updates

LocUS steers LLMs by restricting edits to vocabulary subspaces and sparse heads

Token-level certainty predicts LLM question difficulty better than response correctness

Value-guided search improves math reasoning during RL training

Learned context curation boosts LLM catalog search accuracy

External randomness helps stronger language models produce more diverse responses

Correlated reasoning paths limit the gain from adding more LLM reasoning branches.

Attention distillation complements token-level supervision for reasoning models

Latent feedback helps Transformers carry information across generation steps

Language models can manage their own context more efficiently

Credit assignment that prunes redundant steps improves reasoning accuracy and efficiency

A model monitors live time series while generating responses concurrently

Companion model sharpens LLM confidence without harming task performance

Training a concept generator improves LLM reasoning over repeated sampling.

Ablating individual constraints improves distillation for complex instruction following

Coordinated solution plans make parallel reasoning attempts less redundant

Hidden-state trajectories help diffusion language models escape wrong answers

Token-level teacher routing beats prompt-level routing in multi-teacher distillation

Reasoning makes language models repeat the same wrong answers

Diverse supervised reasoning traces improve generalization after reinforcement learning

Distributed grounding preserves reasoning across million-token language model contexts

Reasoning improves when latent next thoughts remain a distribution

Structured reasoning framework improves LLM game-playing efficiency and payoff

LLM pipelines can lose 40.5 accuracy points between stages

Post-training alignment makes language models more confidently wrong

Co-evolving teacher and student models improves mathematical reasoning

Eight-stage post-training recipe improves an open 106-billion-parameter language model

Teaching Models Confidence Can Shorten Reasoning Without Explicit Stop Training

Selecting evidence before summarizing improves long-context reasoning efficiency

Structural guidance improves language models’ long-horizon reasoning under sparse rewards

Self-play on generated programs produces transferable language-model capabilities

Formal logic solvers can audit and improve language-model reasoning chains

Hybrid backbones can efficiently adapt into diffusion language models

Looped flows enable deeper reasoning by chaining denoising steps

Sparse mixture-of-experts models overfit repeated training data faster

VikingRAG Cuts Token Use for Retrieval Over Structured Documents

Hierarchical memory trees improve long-context reasoning without model training

Discrete diffusion accelerates language models while preserving their autoregressive distribution

Distillation Before Reinforcement Learning Improves Reasoning Model Post-Training

Graph Machine architecture uses dynamic sparse routing to handle large state efficiently

Frontier LLMs design competitive algorithms for well-specified operations problems

Self-evolving loop synthesizes high-quality multimodal training data

Sliding reasoning windows make long test-time scaling 3x faster

Parallel trials and subtasks cut latency in difficult LLM reasoning

Learning-rate-to-parameter-norm ratio predicts language model pretraining loss

Joint training helps small language models create and use tools

A 2.8-trillion-parameter open MoE model approaches frontier performance.

Probabilistic scoring unlocks verification as a new scaling axis for LLMs