diffusion language model
Diffusion language models generate text by starting with random noise and gradually refining it into meaningful sentences, much like an image being slowly revealed from static. Unlike traditional models that write one word at a time from left to right, these models can predict many words in parallel, which can be faster but often requires many refinement steps to get good quality.
The papers you are given all tackle the main weakness of diffusion language models: they need many steps to produce high-quality text. Some introduce ways to train smaller student models that mimic the teacher with far fewer steps, for example by using distillation or a two-level chain where a compact latent vector carries information across steps. Others design special noise schedules or adaptive computation so that easy words are decided quickly while hard words receive more refinement. A common thread is that these techniques drastically reduce the number of required steps, sometimes by an order of magnitude, while matching or even beating autoregressive models on reasoning and generation benchmarks. This makes diffusion language models much more practical for real-world use.
- Papers
- 28
- Released code
- 1
- First seen
- Sept 2026
- Latest
- Nov 2026
28 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Diffusion model fairness shifts with sampling settings, not just weights.
Social bias in diffusion language models varies substantially with sampling configuration, making single-setting evaluations unreliable.
Oct 2026 - 2Guided diffusion enables trustworthy synthetic text with formal privacy guarantees
Guided Diffusion steers diffusion language models toward target semantic embeddings to generate synthetic text with differential privacy and high utility.
Oct 2026 - 3Language models using diffusion can run 2.78 times faster
Diffusion language models run faster when the sparse attention mask is refreshed periodically during denoising and each attention head gets its own budget.
Nov 2026
How fast it is moving
Often studied with
test-time scalingAllocating extra computation during inference to improve output quality.3 togetherunified multimodal understanding and generationA single end-to-end model processes and generates multiple data types (e.g., images, text) with equal competence.2 togetherspeculative decodingA method that uses a lightweight model to propose token sequences that are then verified in parallel by a larger model.2 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Big Techcs.CL
Diffusion model fairness shifts with sampling settings, not just weights.
Social bias in diffusion language models varies substantially with sampling configuration, making single-setting evaluations unreliable.
Institute of Science Tokyo, University of Liverpool · Oct 2026
- Big Techcs.CL
Language models using diffusion can run 2.78 times faster
Diffusion language models run faster when the sparse attention mask is refreshed periodically during denoising and each attention head gets its own budget.
University of Washington, University of Waterloo · Nov 2026
- Big Techcs.CR
Guided diffusion enables trustworthy synthetic text with formal privacy guarantees
Guided Diffusion steers diffusion language models toward target semantic embeddings to generate synthetic text with differential privacy and high utility.
NVIDIA, Carnegie Mellon University · Oct 2026
- Independentcs.CL
AI agents cut computation by a fifth while keeping accuracy competitive
A metacognitive controller reduces diffusion language agent inference FLOPs by 20.6% without losing accuracy.
Oct 2026
- Chinese Techcs.LG
Diffusion models learn better short-budget reasoning from their longer-window trajectories
Masked diffusion language models can be improved by distilling their own large-window reasoning into small-window decoding, boosting accuracy and shortening responses.
The Hong Kong University of Science and Technology (Guangzhou), Zhejiang University · Oct 2026
- Big Techcs.LG
Diffusion language models are cheaper to train than the widely cited 16x compute gap suggests
The widely cited 16x compute gap between diffusion and autoregressive language models shrinks to as low as 3.0x with modern training methods.
Cornell University, Cornell Tech · Oct 2026
- Industrycs.CV
AI models can answer image questions and refine pictures they generate
A unified discrete diffusion LLM can work for both understanding and generating images, using self-refinement and cross-paradigm distillation to match specialized models.
Oct 2026
- Big Techcs.LG
Diffusion language models enforce task-level security against prompt injection
CoDA prevents prompt injection by enforcing task-level security policies within diffusion language model denoising.
University of Virginia, University of Oxford · Oct 2026
- AI Startupcs.LG
On-policy residual training helps diffusion language models decode faster
Residual context for diffusion language models must be learned on-policy with an entropy-contraction objective to improve speed and quality.
Together AI, University of Illinois Urbana-Champaign · Oct 2026
- Chinese Techcs.CV
Multiple visual codes let one diffusion model understand and generate images
A diffusion language model using multi-codebook discrete visual tokens unifies image, video, and 3D understanding with generation and editing in one backbone.
Shanghai AI Laboratory, Shanghai Jiao Tong University · Oct 2026
- Big Techcs.CL
Masked diffusion language models improve by encoding position priority as hierarchy levels
Masked diffusion language models that assign each token position a hierarchy level and noise schedule improve generation perplexity and structured problem-solving accuracy.
Indian Institute of Technology Delhi, IBM Research · Oct 2026
- Big Techcs.AI
Adaptive token computation narrows diffusion language models’ quality gap
Diffusion language models can match autoregressive quality by allocating more recurrent compute to hard tokens while easy tokens commit early.
University of Illinois Chicago, Amazon AGI · Oct 2026
- Big Techcs.CL
Reconstructing text under safety labels produces more reliable guard models
LLaDA-Guard scores text under each safety label hypothesis and classifies based on which label better explains the content.
University of Neuchâtel, Delft University of Technology · Sept 2026
- Top Universitycs.LG
Diffusion models beat autoregressive LLMs on globally constrained problems
Blackboard intelligence from diffusion language models surpasses autoregressive models on globally constrained problems by using mean confidence to guide search and revision.
Seoul National University, Harvard University · Oct 2026
- Big Techcs.CL
Continuous diffusion language models match discrete peers while enabling smoother control
Sigma scales continuous diffusion language modeling to 3B and 8B parameters, matching comparable discrete models while enabling controllable and low-step generation.
NVIDIA, Cornell University · Oct 2026
- Big Techcs.LG
Training on generation trajectories makes masked diffusion language models more efficient
PUMBA trains masked diffusion models on inference-like trajectories, matching autoregressive quality while using fewer function evaluations.
EPFL, Georgia Institute of Technology · Oct 2026
- Big Techcs.LG
Keeping token uncertainty improves discrete diffusion generation and fast sampling
Simplex Diffusion Models represent beliefs over categories on the probability simplex, mitigating information collapse and achieving competitive performance on text and code generation.
Google DeepMind, EPFL · Oct 2026
- Big Techcs.AI
Diffusion language models enable faster plan generation and repair for long-horizon agents
Diffusion language models can generate and repair structured plans by infilling, achieving nearly double the repair success of autoregressive models.
University of Texas at Austin, Amazon · Oct 2026
- Big Techcs.LG
Semi-autoregressive continuous diffusion models match discrete baselines and enable KV caching.
Continuous diffusion language models with semi-autoregressive position-dependent noise schedules achieve state-of-the-art likelihood and benchmark performance while enabling variable-length generation and KV caching.
NVIDIA, Cornell University · Oct 2026
- Big Techcs.CL
Coupling continuous latent with token feedback improves diffusion language models
Coupling continuous latent trajectories with token readout and feedback improves diffusion language models on reasoning and language modeling.
University of Illinois Urbana-Champaign, Amazon.com, Inc. · Oct 2026
- Chinese Techcs.CL
Looped diffusion drafting improves decoding without extra model components
D-Loop adds intra-block causal conditioning to diffusion drafting, improving acceptance length and decoding speed without extra model components.
City University of Hong Kong, The Chinese University of Hong Kong · Oct 2026
- Big Techcs.CL
Hidden-state trajectories help diffusion language models escape wrong answers
Hidden-state trajectories enable detecting and repairing stable-but-wrong lock-in in diffusion language models, yielding accuracy gains of 2.21-5.37 percentage points.
Rochester Institute of Technology, NVIDIA Corporation · Sept 2026
- Big Techstat.ML
Average generator yields 16x faster discrete diffusion language model sampling
The Discrete Average Generator achieves state of the art generative perplexity on OpenWebText with 16x fewer generation steps.
University of California, Los Angeles, Google · Oct 2026
- AI Startupcs.LG
Distributional distillation cuts diffusion language model sampling steps by half or more
Two distribution matching distillation methods reduce network evaluations for continuous diffusion language models by up to 49% and 20% at similar generation quality.
Inria, PSL Research University · Oct 2026
- Academiccs.CL
Hybrid backbones can efficiently adapt into diffusion language models
Hybrid-attention diffusion models from Qwen3.5 adapt faster and decode in any order and in parallel.
University of Texas at Austin · Sept 2026
- Big Techcs.LG
A language model using only self-conditioning outperforms existing diffusion models
A continuous generative model can be built purely on self-conditioning by propagating the expected embedding across denoising steps and decoding once at the end.
University of Basel, Microsoft AI · Oct 2026
- Big Techcs.LGcode
Future-aware distillation corrects teacher-student information mismatch in block diffusion language models
Correcting an autoregressive teacher's next-token distribution with visible future context improves distillation of block diffusion language models from pretrained AR models.
Tsinghua University, MIT · Oct 2026
- AI Startupcs.CL
Coarse-to-fine text generation matches diffusion quality at much higher speed
Next-Concept Modeling generates text by predicting all codes within each hierarchical level in parallel, achieving faster generation with fewer steps.
Stanford University, Together AI · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.