← All concepts

multi-token prediction

For an AI that predicts one word at a time, multi-token prediction means guessing several future words in one go. Normally, the model learns to do this during its initial enormous training run. The idea is that predicting a chunk of text at once forces the model to plan ahead, which can make it better at tasks like reasoning, and also speeds up text generation because the model can accept or reject its guesses in bulk.

The papers show two main trends. First, the prediction heads used for guessing future tokens can be taught after the main model is trained, using far less data than previously needed. Second, the same model that does the main prediction can also serve as its own guesser for faster generation, either through clever architecture changes that reuse the main network, or by splitting the workload between a phone and a server. All approaches boost generation speed and often improve reasoning quality.

Papers
6
Released code
0
First seen
Oct 2026
Latest
Nov 2026

6 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Shared language-model layers make multi-token prediction cheaper and more direct

    ECHO reuses the backbone's final layer for multi-token prediction and uses dynamic depth sampling, improving performance and enabling efficient speculative decoding.

    Oct 2026
  2. 2One-line fix makes language models learn better and decode faster for free

    Soft-MTP improves multi-token prediction training by filtering out noisy gradient signals, boosting performance and inference speed without added cost.

    Oct 2026
  3. 3Language models can generate text up to three times faster

    Multi-token prediction via a shared drafting head achieves up to 3.1x speedup for LLM inference on servers and up to 2.2x on smartphone GPUs.

    Nov 2026

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.

multi-token prediction · Research concepts | Zotpaper