speeds up language model inference by having a lightweight drafter propose tokens that a target model verifies. However, existing drafters become more expensive as context grows, undermining their advantage. LongSpark introduces a block-diffusion drafter that extracts fixed-size, multiscale views from the target's pass, making its computational cost constant regardless of prefix length. This eliminates the need for a growing persistent state in the drafter.
Evaluated on reasoning, coding, and dialogue benchmarks across multiple model scales, LongSpark achieves the highest end-to-end throughput with speedups of 1.88x to 2.13x, which improve with larger targets. It reduces the drafter's context state by several orders of magnitude while delivering the lowest time per output token on long-context tasks. Notably, it outperforms state-of-the-art drafters even when those achieve longer accepted lengths, proving that bounded overhead matters more than marginal acceptance gains.