6B.
Centering activations enables calibration-free 4-bit diffusion transformer quantization
CentriQ removes token-wise activation means before rotation, restores them exactly, and achieves W4A4 image quality comparable to a calibrated method.
Top University
Nataša Jovanović · Mathieu Salzmann · Saqib Javed
Tenstorrent · EPFL
Research Digest··3 min read
Jovanović, Salzmann and Javed identify why data-free Hadamard rotation performs poorly when quantizing diffusion transformers: adaptive layer normalization introduces a token-specific mean that rotation concentrates into a few range-dominating coordinates.
Why this paper
From EPFL and Tenstorrent
In one line
Exact per-token mean centering enables calibration-free 4-bit DiT quantization to match calibrated SVDQuant across three model families.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§