Language models using diffusion can run 2.78 times faster

Zeng and colleagues report faster generation without retraining by periodically updating which token connections the models compute and assigning different computation budgets to different attention heads.

Big Tech

Zhichen Zeng · Xichong Zhang · Junpan Wu · Yifei Zuo · Chi-Chih Chang · Jiayi Wang · +4 more

University of Washington · University of Waterloo · University of Wisconsin, Madison · Northwestern University · Cornell University

Research Digest·3 November 2026·2 min read

Illustration: Language models using diffusion can run 2.78 times faster

In one line

Diffusion language models run faster when the sparse attention mask is refreshed periodically during denoising and each attention head gets its own budget.

Before you read this

Two ideas this paper builds on

diffusion language modelA language model that generates text by iteratively denoising a latent or discrete representation.28 papers →sparse attentionSelectively attending to only the most relevant positions or tokens to reduce computational overhead.39 papers →

Read it as

Chatbots normally write one word at a time and can cache earlier work. instead draft a whole passage and refine every word at once over repeated passes, so nothing can be cached and each pass redoes the full attention calculation, which is slow on long text.

Keeping only a chosen fraction of word-to-word links, a sparse mask, is one fix. This work refreshes that mask at a few checkpoints and reuses it in between, instead of fixing it once, and gives each attention head, an independent sub-channel, its own share of links.

The payoff: a 2.78x speedup at long context with most links dropped, scoring better than earlier sparse methods.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

diffusion language model sparse attention