Research · Efficiency & systems
H-SVDQuant improves 4-bit quantization of diffusion transformers with low-rank branches
Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers
A unified framework jointly modeling weight and activation quantization error achieves better performance with rank 4 than existing methods with rank 32.
MMLU: 68.17% accuracy (as reported)
The University of Hong Kong, Huawei Technologies Co., Ltd.