Speculative Decoding Optimization
Methods for optimizing speculative decoding including multi-token prediction, drafting strategies, and device-cloud coordination.
8 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
How this thread developed
Oct 2026 · Renmin University of China, National Key Lab for Novel Software Technology, Nanjing University
Tree-trained draft branches make speculative decoding accept longer continuations
tree of draft branches extends acceptance length beyond single continuation
1 further paper
Oct 2026 · Samsung Research
Hybrid drafters split depth and breadth to cut mobile-cloud GPU costs
Proposes HyCo, a hybrid drafting scheme that splits depth and breadth for device-cloud speculative decoding.
2 further papers
Oct 2026 · Tianjin University of Technology, Baidu Inc.
Speculative decoding throughput gains are conditional on workload and serving conditions
Systematically measures when added acceptance length repays computation across workloads and serving conditions, informing drafter recipe choices.
Oct 2026 · University of California, San Diego, Massachusetts Institute of Technology
Jointly co-compiling draft and target models boosts speculative decoding speed
Introduces Conjugate Compilation to jointly co-compile draft and target models, improving draft acceptance and decoding speed.
Oct 2026 · ERNIE Team, Baidu, Independent Researcher
Shared language-model layers make multi-token prediction cheaper and more direct
presents ECHO for training multi-token prediction models with shared layers, making draft-model training cheaper and more direct.
5 of 8 papers shown