Continuous diffusion language models match discrete peers while enabling smoother control
Yang et al. introduce Sigma, a blockwise language model that generates through continuous diffusion rather than predicting tokens autoregressively or denoising categorical states. Across math and coding evaluations, the authors report performance comparable to similarly sized discrete diffusion and autoregressive models, while continuous trajectories provide direct control over output quality, diversity and inference steps.
