Conjugate Compilation treats speculative decoding as a cross-model compilation problem, expanding the compiler's optimization boundary to include both models' co-located execution.
Jointly co-compiling draft and target models boosts speculative decoding speed
Self-improving agentic system uses measured co-located execution to optimize both models together, achieving geometric-mean 1.37x per-round speedup
Big Tech
Annabelle Sujun Tang · Rohan Mahapatra · Christopher Priebe · Hanyang Xu · Manya Ghobadi · Christopher Rossbach · +1 more
University of California, San Diego · Massachusetts Institute of Technology · Systalyze · University of Texas at Austin · Microsoft
Research Digest··2 min read
The authors introduce Conjugate Compilation, a method that reframes speculative decoding as a cross-model compilation problem by jointly optimizing the small draft model and the larger target model within their shared execution on a single GPU.
§