What they did

The authors developed AsymSpec, a speculative-decoding framework in which a lightweight drafter receives the full input but the larger verifier sees a compressed version. Speculative decoding normally accelerates generation by having a small model propose tokens that a larger model checks, with both models using identical context.

AsymSpec instead combines the models’ token scores through contrastive δ-fusion. A gate based on disagreement between the models controls acceptance, aiming to use information preserved by the full-context drafter without destabilizing verification. The authors evaluated the approach on four agentic capabilities and two end-to-end agent benchmarks.

Key findings

• AsymSpec achieved approximately 90% of full-context accuracy on average despite giving the large verifier only compressed context.
• On isolated text capabilities, it produced 1.3–1.7× higher throughput.
• Compute cost on those isolated capabilities fell to 0.2–0.3× the comparison level reported by the authors.
• The largest benefits appeared when context compression removed information important for reasoning.

Why it matters

The results suggest that context compression need not force all components of an agent pipeline to discard the same information. Giving a cheap drafting model full context may recover useful signals while limiting the cost of running a much larger model over long retrieval, tool-use, and conversation histories.

Caveats

The reported speed and compute improvements are specifically stated for isolated text capabilities, so their magnitude in complete agent deployments remains unclear. The method also recovers about 90%, rather than all, of full-context accuracy on average; its effectiveness may depend on the compression method, model pairing, and how much task-critical information the compressed view loses.