The authors decomposed multimodal self-attention into four interaction types: text-to-text, text-to-vision, vision-to-text and vision-to-vision.
Manipulating queries and keys enables training-free MM-DiT editing
QK-Edit preserves visual structure while applying text-guided changes by selectively replacing image or video components in attention queries and keys.
Chinese Tech
Tiancheng Shen · Zilong Huang · Xiangtai Li · Zhijie Lin · Jiyang Liu · Yitong Wang · +3 more
UC Merced · ByteDance Seed · ByteDance Intelligent Creation · CUHK
Research Digest··2 min read
Shen and colleagues analyze how text and visual tokens interact inside multimodal diffusion transformers, then use those observations to build a training-free editor for images and videos.
Why this paper
From ByteDance Seed and 3 others
In one line
QK-Edit manipulates query-key attention in MM-DiTs for training-free text-guided image and video editing.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (3 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§