Discrete are used to generate text and code by gradually corrupting then denoising data. Sampling from them often requires many steps and can be inconsistent. An analogous deterministic sampler exists for continuous data, but not for discrete.
This work introduces exponential residual flows, which add a continuous variable to each token. These variables evolve according to deterministic equations, and the discrete token changes when the variable hits zero. This creates a deterministic sampler that preserves the true probabilities of the original diffusion model.
The method works for common discrete diffusion types and yields empirical improvements on text and code generation benchmarks. It also supports consistency distillation, which can further speed up sampling.