The authors combine a causal autoregressive transformer with a lightweight diffusion head.
Reasoning improves when latent next thoughts remain a distribution
Autoregressive Thought Flow samples alternative continuous thoughts with a diffusion head, enabling compact, variable-length reasoning paths.
Big Tech
Yang Li · Yi Wang · Shiyuan Huang · Yang Liu · Hao Wang · Chengzhi Mao
Rutgers University · Amazon · University of Illinois at Urbana-Champaign
Research Digest··2 min read
Li and colleagues introduce Autoregressive Thought Flow, which represents each next latent thought as a multimodal distribution rather than a single predicted state.
Why this paper
From Amazon and 2 others
In one line
Modeling each latent next thought as a multimodal distribution improves compact mathematical reasoning and preserves diverse solution paths better than point predictions.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§