Video generation models use a step called attention to build relationships between all video pieces, but this step becomes extremely slow as video size grows because it checks every piece against every other piece. This quadratic scaling can make high-resolution or long videos impractical.
The authors propose AB-Attention, a method that estimates the full attention without computing all pairs. It groups video pieces into blocks and scores only a sampled subset of these blocks, then reweights their contributions to get an unbiased estimate. This avoids simply dropping blocks, which can lose important information.
On two major video generation models, the method skips 89% to 92% of the attention computation and achieves up to 4.4 dB higher quality on a standard measure (PSNR) compared to existing methods. This allows generating longer or higher-resolution videos with the same hardware, without sacrificing quality.