With Qwen3-VL-8B as the target and a DFlash-style block drafter, they trained three objectives: cross-entropy on corpus tokens, token-level distillation from the target, and the same distillation gated to positions that would survive verification.
Draft-training objective gains shrink when target regenerates training responses
Token-level distillation and gated distillation add large acceptance gains on external responses, but most of that gain vanishes after the target greedily regenerates the corpus.
Academic
Jungseob Lee · Hogun Seong · Euijun Choi · Sugyeong Eo
Korea University · Yonsei University Mirae Campus
Research Digest··2 min read
The authors compare three draft-training objectives on external versus target-regenerated responses for a vision-language speculative decoding pipeline.
§