quantization-aware training
Quantization-aware training means teaching a neural network to tolerate having its numbers rounded to fewer bits while it is being trained. Instead of training in full precision and then compressing, engineers insert simulated 'fake' quantization layers so the model learns to produce weights and activations that survive coarser representation. This mimics deployment hardware without requiring it, making low-precision inference practical.
The papers show this works across 2-bit to 4-bit formats, often beating post-training compression. Success depends on matching training to the target format, such as choosing which parts to quantize, using rotations, lattices, or careful rounding. They also report substantial memory savings and throughput gains, and warn that short experiments can mislead; the right combination of techniques matters.
- Papers
- 24
- Released code
- 2
- First seen
- Oct 2026
- Latest
- Nov 2026
24 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Scalar two-bit weight quantization can preserve language model reasoning accuracy
Strictly scalar two-bit quantization with sensitivity-based bit allocation, codeword search, scale recalibration, and reasoning-based selection preserves LLM reasoning.
Oct 2026 - 2Small vision-language AI models can be compressed fourfold with only 1% accuracy loss
ProQuant makes low-bit W4A8 quantization of small multimodal LLMs nearly lossless by progressively restoring multimodal understanding before language modeling.
Nov 2026 - 3Language models can learn compressed weights with less training memory
ResQAdam reduces the memory cost of quantizing large language models while staying close to AdamW's accuracy.
Nov 2026
How fast it is moving
Often studied with
post-training quantizationQuantizing pre-trained neural network weights and activations to10 togetherkv cache compressionReducing the size of the key-value cache in transformers via selective retention, low-rank updates, or adaptive compression.6 togetherlarge language modelA neural network trained on massive text corpora to generate coherent text and perform diverse language tasks.5 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Top Universitycs.LGcode
Language models can learn compressed weights with less training memory
ResQAdam reduces the memory cost of quantizing large language models while staying close to AdamW's accuracy.
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), King Abdullah University of Science and Technology (KAUST) · Nov 2026
- Top Universitycs.CV
Small vision-language AI models can be compressed fourfold with only 1% accuracy loss
ProQuant makes low-bit W4A8 quantization of small multimodal LLMs nearly lossless by progressively restoring multimodal understanding before language modeling.
The Hong Kong University of Science and Technology, Northwestern Polytechnical University · Nov 2026
- Big Techcs.AI
Scalar two-bit weight quantization can preserve language model reasoning accuracy
Strictly scalar two-bit quantization with sensitivity-based bit allocation, codeword search, scale recalibration, and reasoning-based selection preserves LLM reasoning.
Microsoft Applied Sciences Group · Oct 2026
- Academiccs.LG
Language models can be trained for compression nearly 13 times faster
SQuAT improves low-bit LLM compression by applying separate straight-through estimators for shape and gain in vector quantization.
University of Copenhagen, Qualcomm AI Research · Oct 2026
- Industrycs.AI
Language models can learn nearly as well with four-bit numbers
MXFP4 training closes validation loss gap to BF16 to 0.007 using 2D quantization, RHT, SR, and weight de-oscillation.
Advanced Micro Devices, Inc. · Oct 2026
- Big Techcs.CL
Selective high-precision layers close the accuracy gap in four-bit training
A recipe combining selective high-precision layers, unbiased gradient estimation, and improved scale selection achieves the lowest FP4 training loss gap to BF16 at 8B-30B scale.
NVIDIA, Institute of Science and Technology Austria · Oct 2026
- Chinese Techcs.AI
E8 lattice quantization preserves model accuracy at two-bit precision
E8-QAT delivers accurate 2-bit LLM weights by projecting eight-weight groups onto an implicit E8 lattice without explicit codebook storage or search.
Peking University, City University of Hong Kong · Oct 2026
- Big Techcs.AI
Canonical training improves language models under joint low-bit compression
CanonQ unifies weight, activation, and KV cache quantization using frozen reference codebooks and joint training.
Meta AI · Oct 2026
- Academiccs.LGcode
Neural network precision floor follows a depth law predicted by amplification
The precision floor of neural networks follows a power law in depth, predicted by predictive amplification with a constant collapse constant.
Mutah University · Oct 2026
- Chinese Techcs.LG
Aligning FP4 training and rollout stabilizes reinforcement learning
TRACE aligns FP4 training and rollout quantization for MoE RL, matching BF16 rollout performance while accelerating rollout by up to 5.4x.
Alibaba Token Hub, Alibaba Group, Ohio State University · Oct 2026
- AI Startupcs.LG
Training models for deployment preserves quality under four-bit quantization
QATFactory enables quantization-aware training that improves deployed LLM quality over post-training quantization for NVFP4, MXFP4, and Q4_K formats.
Together AI, University of Illinois Urbana-Champaign · Oct 2026
- Big Techcs.CV
Video generators can nearly halve history storage while improving reconstruction quality
A compression scheme for cached attention state in autoregressive video models preserves generation quality better by tuning Keys to match downstream outputs rather than to reconstruct the raw cache.
Zhejiang University, NVIDIA · Nov 2026
- Big Techcs.LG
Sharing random numbers speeds integer training when rounding errors are corrected
S3R-EC matches independent stochastic rounding's quality in LLM pre-training while accelerating INT8 training by 1.11x.
Google · Oct 2026
- Big Techcs.CL
Training on self-generated answers improves reasoning in low-bit language models
OnQ recovers reasoning in low-bit LLMs by training quantized students on their own generated responses with candidate-wise Dynamic KL, outperforming fixed-response distillation.
University of Michigan, Ann Arbor, University of California, Santa Barbara · Oct 2026
- Big Techcs.AI
Vector codebooks preserve useful language models at roughly two bits
At 2-bit weights, vector codebooks with jointly chosen calibration and recovery data keep 89-97% of full-precision QA accuracy where scalar PTQ fails.
Applied Sciences Group (ASG), Microsoft, Microsoft Research · Oct 2026
- Academiccs.LG
Compressed AI memory can be scored directly, speeding generation 2.5x
GaugeRoPE enables direct latent key-value attention without per-token reconstruction, achieving 5.85× KV compression on Mistral-7B.
National University of Defense Technology · Nov 2026
- Top Universitycs.LG
Language models can lag 50 to 150 updates behind training
Standard bf16 rounding silently discards most RL updates, creating a hidden lag that unbiased rounding eliminates.
Stanford University · Nov 2026
- Top Universitycs.LG
Language models preserve more retrieval accuracy with 97-fold smaller caches
Extreme KV-cache compression is practical using finite scalar quantization, achieving accurate retrieval and fast decoding.
The University of Edinburgh · Nov 2026
- Independentcs.AI
Native FP4 KV cache keeps long-context LLM accuracy while doubling throughput
BitDecoding-FP4 enables accurate native FP4 KV cache decoding, achieving up to 3.0x throughput over BF16 KV.
Nov 2026
- Big Techcs.LG
One compact model embeds text, images, audio and video together
EmbeddingGemma 2 unifies text, images, audio, and video into a single embedding space, outperforming models two to twelve times larger.
Google · Oct 2026
- Research Labcs.AI
4-bit compression speeds up LLM training by 56 percent with minimal accuracy loss
FP4Quant compresses model-parallel LLM communication to 4-bit floats, cutting quantization overhead and improving end-to-end training throughput by up to 1.56x.
University of Chinese Academy of Sciences, Institute of Computing Technology, Chinese Academy of Sciences · Nov 2026
- Industrycs.SD
Open-vocabulary speech recognition runs on a $5 microcontroller without a neural accelerator
Oído brings open-vocabulary English speech recognition to a $5 microcontroller without a neural accelerator.
Lokutor · Oct 2026
- Big Techcs.CL
Hybrid drafters split depth and breadth to cut mobile-cloud GPU costs
HyCo reduces server GPU time by 38.6% using a hybrid drafter with MTP head and SLM on mobile.
Samsung Research · Oct 2026
- Independentcs.LG
A compact AlphaZero-style chess system reaches 3,251 Elo after 2.5 days on a single GPU node
An engineered AlphaZero-style chess system achieves 3251 Elo in 2.5 days on one 8-GPU node.
Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.