Agent framework auto-tunes video diffusion for 2x speedup

What they did

The authors built an agentic acceleration framework that automatically tunes five orthogonal video diffusion acceleration techniques for a specific deployment target. For a given model, hardware platform, and inference configuration (e.g., resolution, video duration), parallel "skill agents" independently optimize each technique's implementation (cache, sparse attention, token pruning, quantization, kernel fusion). An "agent integrator" then composes these into a global acceleration stack, and a human validator checks generation quality. The workflow was instantiated on three video models of varying scale: 64B Cosmos3-Super, 22B LTX-2.3, and 2B SANA-Video. No additional training was required; all optimizations are applied post-training.

Key findings

The composed acceleration stack achieves >2x end-to-end inference speedup across all three models on target hardware (NVIDIA GPUs), with near-lossless quality as measured by the VBench video benchmark.
The agent-based approach avoids manual performance engineering: the best configuration for one (model, hardware, configuration) triplet often fails to transfer to another, but the framework discovers instance-specific recipes automatically.
Each of the five techniques contributes meaningfully; the framework demonstrates that combining them yields compound gains without quality degradation, whereas naive stacking often hurts.

Why it matters

Video diffusion models are increasingly expensive to run as they scale, limiting practical deployment. Sol provides a systematic, automated way to accelerate inference without retraining, reducing the human effort barrier. By treating acceleration as an instance-specific optimization problem and solving it with an agentic design, the paper offers a pragmatic path to deploying large video models in latency-sensitive or cost-sensitive applications.

Caveats

The experiments are limited to three models from a single model family (NVIDIA's Cosmos and SANA, plus LTX-2.3) and target NVIDIA GPU hardware. The framework's generality across other architectures (e.g., non-diffusion video models, mobile or edge hardware) is not evaluated. The human validator step, while minimal, still requires human intervention for quality checks. The paper does not compare against an exhaustive set of prior acceleration methods or study the agent's tuning time vs. manual tuning.