The authors analyzed the hidden states of a standard DiT during training, measuring cosine similarity across layers and gradient flow.
Structured residual connections improve diffusion transformers by enabling adaptive cross-layer information retrieval
The proposed method replaces uniform residual streams with selective, learnable pathways, achieving faster convergence and significant FID gains with minimal parameter overhead.
Top University
Yuhe Liu · Xinyin Ma · Gongfan Fang · Songhua Liu · Xinchao Wang
National University of Singapore · Shanghai Jiao Tong University
Research Digest··3 min read
Liu et al.
Why this paper
From National University of Singapore and Shanghai Jiao Tong University
In one line
Structured residual connectivity in Diffusion Transformers yields faster convergence and better FID with negligible added parameters.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§