Discrete diffusion and flow-matching models generate tokens by evolving a continuous-time Markov chain (CTMC), but few-step sampling is inaccurate because coordinates are updated independently.
Average generator yields 16x faster discrete diffusion language model sampling
Ouyang et al. extend MeanFlow to continuous-time Markov chains, defining an average generator with a self-consistency identity for few-step generation.
Big Tech
Yidong Ouyang · Zhengyan Wan · Themis Haris · Tian Tan · Liqian Peng · Henry Li · +5 more
University of California, Los Angeles · Google
Research Digest··3 min read
Ouyang and colleagues introduce the Discrete Average Generator, a MeanFlow-style construction for discrete state spaces.
Why this paper
From Google and University of California, Los Angeles
In one line
The Discrete Average Generator achieves state of the art generative perplexity on OpenWebText with 16x fewer generation steps.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§