← All concepts

quantization-aware training

Quantization-aware training means teaching a neural network to tolerate having its numbers rounded to fewer bits while it is being trained. Instead of training in full precision and then compressing, engineers insert simulated 'fake' quantization layers so the model learns to produce weights and activations that survive coarser representation. This mimics deployment hardware without requiring it, making low-precision inference practical.

The papers show this works across 2-bit to 4-bit formats, often beating post-training compression. Success depends on matching training to the target format, such as choosing which parts to quantize, using rotations, lattices, or careful rounding. They also report substantial memory savings and throughput gains, and warn that short experiments can mislead; the right combination of techniques matters.

Papers
24
Released code
2
First seen
Oct 2026
Latest
Nov 2026

24 papers in the last two months, against 0 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Scalar two-bit weight quantization can preserve language model reasoning accuracy

    Strictly scalar two-bit quantization with sensitivity-based bit allocation, codeword search, scale recalibration, and reasoning-based selection preserves LLM reasoning.

    Oct 2026
  2. 2Small vision-language AI models can be compressed fourfold with only 1% accuracy loss

    ProQuant makes low-bit W4A8 quantization of small multimodal LLMs nearly lossless by progressively restoring multimodal understanding before language modeling.

    Nov 2026
  3. 3Language models can learn compressed weights with less training memory

    ResQAdam reduces the memory cost of quantizing large language models while staying close to AdamW's accuracy.

    Nov 2026

Who is working on it

NVIDIA 2Google 2

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.