HC-DLM is a diffusion framework for discrete sequences that uses a two-level chain: a continuous latent trajectory persists across steps, and tokens are read out from the latent at each step and fed back as conditioning for the next latent update. This hierarchical coupling addresses the token-independence bottleneck in parallel decoding by routing cross-token dependence through a compact latent. On structured reasoning tasks (Sudoku, Countdown) and language modeling (LM1B), HC-DLM improves over discrete and continuous diffusion baselines at matched model size, in puzzle accuracy and generative perplexity respectively. The framework applies to both globally constrained generation and unconditional language modeling.
Coupling continuous latent with token feedback improves diffusion language models
HC-DLM models the reverse process as a two-level chain where tokens are read out from a shared latent at each step and fed back to condition the next latent transition, tying the continuous state to valid discrete configurations.
Big Tech
Hui Ren · Zihan Li · Chang Liu · Huidong Liu · Alexander Schwing
University of Illinois Urbana-Champaign · Amazon.com, Inc.
Research Digest··3 min read
Thread:Diffusion Language Models
In one line
Coupling continuous latent trajectories with token readout and feedback improves diffusion language models on reasoning and language modeling.
Before you read this
An idea this paper builds onRead it as
Ren et al. propose Hierarchical Continuous Diffusion Language Models (HC-DLM), a generative framework that combines a continuous latent trajectory with discrete token readouts at every denoising step.
Ren et al. introduce HC-DLM, a diffusion model for discrete sequences with a hierarchical reverse process.
How it works
stage by stageThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.