Language models generate 5–8% faster by keeping draft alternatives separate

Huang and colleagues show that drafting several coherent continuations avoids wasted verification work and outperforms an identically trained single-alternative drafter across four Qwen models.

AI Startup

Haoyu Huang · Jialiang Kang · Linlin Yang · Xiao Yang · Baochang Zhang

Beihang University · Canva Research · Communication University of China

Research Digest·11 November 2026·2 min read

Illustration: Language models generate 5–8% faster by keeping draft alternatives separate

In one line

Multi-Future Drafting avoids chimeras in speculative decoding by using mode embeddings, delivering up to 7.8× speedup.

Before you read this

Two ideas this paper builds on

speculative decodingA method that uses a lightweight model to propose token sequences that are then verified in parallel by a larger model.56 papers →verificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.323 papers →

Read it as

Generating text with a language model token by token is slow. A speed-up technique, , uses a fast model to draft a whole block of words which a big model verifies in one go. The drafter creates a tree of possible continuations, but the standard way of ranking these paths mixes bits of different plausible continuations together.

This mixing wastes the tree's budget on impossible paths. Multi-Future Drafting (MFD) fixes this by supplying the drafter with several distinct modes (separate storylines to follow) during its single pass. Each mode generates a coherent block of words for a different plausible future.

The verifier then picks the best block. On several large language models, MFD sped up generation by up to 7.8 times and was consistently faster than existing approaches.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

speculative decoding verification block drafting multi-future drafting mixture tree latent variable