(DiTs) are costly for reference-conditioned generation because they repeatedly process reference tokens. The paper observes that reference representations change slowly and the target pays little attention to them, suggesting reuse opportunities. RefAdapt-DiT is a training-free method that uses target query changes and past attention mass to decide when to refresh reference states at each depth. On 4-step MiniMax H3 and 8-step Qwen Image Edit, it achieves speedups of 2.097x and 3.54x respectively while maintaining visual quality. This is interesting because it reduces computation without retraining or modifying the model architecture.
Adaptive reference attention reuse speeds up conditioned diffusion transformers
RefAdapt-DiT uses observed target dynamics to decide when to reuse stale reference keys and values without retraining.
Chinese Tech
Jian Tang · Jiawei Fan · Qiannan Zhou · Qingbin Liu · Jiang Bian · Zang Li
Tencent
Research Digest··2 min read
Thread:Diffusion Language Models
In one line
RefAdapt-DiT adaptively reuses reference computations in diffusion transformers, achieving up to 3.54x speedup on Qwen Image Edit with comparable quality.
Before you read this
An idea this paper builds onRead it as
The authors propose RefAdapt-DiT, a training-free framework that adaptively controls joint attention computation between reference images and target generation in diffusion transformers.
Tang et al. first analyzed reference–target dynamics in (DiTs) for reference-conditioned tasks such as multi-reference image editing and video generation.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.