1-8B-Instruct with Gaussian prediction heads, each trained to represent a future span of a particular length as a fixed-dimensional latent vector.
A learned router chooses when language models should draft farther ahead
CADENCE predicts variable-length token spans as latent vectors, selecting longer drafts only when the current context appears sufficiently predictable.
Big Tech
Sai Sreenivas Chintha · Chidera Biringa · Rishab Sharma · Anusha Agrawal · Prachi Badarayani · Gemma Zhang · +3 more
Microsoft · University of Massachusetts Amherst
Research Digest··2 min read
Chintha and colleagues present CADENCE, a decoding method that lets a language model choose between generating one token normally and proposing a longer continuation in parallel.
§