Low-Bit Quantisation
Methods for compressing large language models to very low bitwidths (e.g., 2 bits) while preserving task accuracy.
8 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
5 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Nov 2026 · Huawei Technologies Co. Ltd.
New method compresses large language models to 2 bits with minimal accuracy loss
This paper is new. TVQ achieves 2-bit vector quantization via block-wise distillation, advancing extreme compression for LLMs.
Nov 2026 · UT Southwestern Medical Center, Emory University
4-bit language models run as integer math on Apple's M5 chips without extra accuracy loss
Shows MXFP4 weights can be converted to INT8 operands for exact W4A8 inference on Apple M5 without accuracy loss.
released code
Nov 2026 · Amazon Web Services
Combining model compression techniques can collapse small language models
Investigates how combining quantization with pruning and fine-tuning can cause performance collapse in small language models, informing robust compression design.
1 further paper
Nov 2026 · Indraprastha Institute of Information Technology, Delhi, TurboML
Larger language models can lose more retrieval ability after compression
Analyzes how compression damages internal computations for retrieval, with models of different scales reacting differently.
2 further papers
Nov 2026 · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), King Abdullah University of Science and Technology (KAUST)
Language models can learn compressed weights with less training memory
Introduces ResQAdam optimizer that reduces training memory for low-bit models by preserving only essential update signals, enabling more efficient low-bit training.
released code
5 of 8 papers shown