The authors move discrete diffusion from individual categories onto the probability simplex, the space of probability distributions over all possible categories.
Keeping token uncertainty improves discrete diffusion generation and fast sampling
Simplex Diffusion Models refine probability distributions over tokens, producing competitive language samples and stronger code-generation results with fewer denoising steps.
Big Tech
Justin Deschenaux · Alexandre Galashov · Andrew Campbell · Li Kevin Wenliang · James Thornton · Arnaud Doucet · +1 more
Google DeepMind · EPFL · UCL Gatsby
Research Digest··2 min read
Deschenaux et al.
Why this paper
From Google DeepMind and 2 others
In one line
Simplex Diffusion Models represent beliefs over categories on the probability simplex, mitigating information collapse and achieving competitive performance on text and code generation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (3 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§