1, supplemented by a theoretical derivation via RoPE-3D logit decomposition and video spatio-temporal redundancy.
Periodic stripe structures in video DiT attention enable efficient sparse computation
The authors identify a diagonal stripe pattern in attention distributions and formalize it into a parameterized mask that unifies sparse patterns, achieving end-to-end speedups of 1.57× and 1.37× on HunyuanVideo and Wan 2.1 over FlashAttention-3.
Chinese Tech
Xingyu Jia · Baole Ai · Ang Wang · Kang Zhao · Yong Li
Alibaba Group
Research Digest··2 min read
The paper reveals that attention layers in video Diffusion Transformers exhibit periodic diagonal stripe structures along temporal and spatial dimensions.
Why this paper
From Alibaba Group
In one line
Parameterized stripe attention exploits recurring video DiT attention patterns to accelerate HunyuanVideo and Wan 2.1 by 1.57× and 1.37× with limited quality loss.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§