Sigma is a blockwise continuous that generates through smooth, 16-dimensional token-embedding trajectories rather than categorical states. Its 3B and 8B variants are warm-started from pretrained autoregressive models and trained to denoise Gaussian-corrupted embeddings while learning the embedding geometry.
Across math reasoning and coding evaluations, Sigma performs competitively with masked diffusion and autoregressive baselines. Classifier-free guidance and score temperature control the quality-diversity trade-off, while embedding regularization and trajectory distillation preserve useful performance with fewer denoising steps. This suggests continuous diffusion can combine parallel generation with more steerable inference trajectories.