Loeschcke et al.
Separate shape and gain gradients bring lattice VQ-QAT to billion-parameter LLMs
Loeschcke et al. decompose weight blocks into direction and magnitude and apply factored straight-through estimators, improving accuracy over joint-STE baselines and matching scalar INT-QAT below 2.5 bits per weight.
Academic
Sebastian Loeschcke · Ivan Koryakovskiy · Tycho F. A. van der Ouderaa · Riccardo Del Chiaro · Mart van Baalen · Paul N. Whatmough · +1 more
University of Copenhagen · Qualcomm AI Research
Research Digest··2 min read
The authors introduce SQuAT, a quantization-aware training method for lattice-based vector quantization of LLM weights, decomposing each weight block into a normalized direction and a scalar gain.
§