The authors evaluated FidelityQ on LLaMA and Qwen models under weight-only quantization and joint quantization of weights, activations and key-value caches.
Two-bit scalar quantization better preserves language model predictions
FidelityQ combines adaptive four-level weight coding with targeted teacher-guided calibration to improve accuracy while retaining simple integer-compatible decoding.
Chinese Tech
Yujie Chen · Weiyan Zheng · Yiwu Yao · Tianchi Hu · Bo Wang · Jiyu Guo · +4 more
The Hong Kong Polytechnic University · Huawei Technologies Ltd. · The University of Hong Kong · Sun Yat-sen University · ETH Zurich
Research Digest··2 min read
Chen and colleagues present FidelityQ, a framework for compressing language-model weights to two bits, giving each weight only four possible reconstruction values.
Why this paper
From Huawei Technologies Ltd. and 4 others
In one line
Two-bit scalar quantization preserves LLM predictive fidelity through mirrored four-level coding and calibration targeted at teacher-student divergence.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§