Quantization reduces the numerical precision used to represent model parameters, cutting memory requirements but potentially damaging reasoning.
Training on self-generated answers improves reasoning in low-bit language models
Across three Qwen3 model sizes at 2- and 3-bit precision, OnQ improves benchmark accuracy while reducing truncated and repetitive responses.
Big Tech
Boyuan Zheng · Jing Xiong · Chenwei Wu · Hui Shen · Zesen Zhao · Hei Ting Una Chan · +5 more
University of Michigan, Ann Arbor · University of California, Santa Barbara · Meta AI
Research Digest··2 min read
Zheng and colleagues introduce OnQ, a training pipeline that helps aggressively quantized language models recover reasoning performance by learning from their own generated responses.
§