Precision Scaling Laws
Investigating the relationship between network depth, parameter perturbations, and the minimum precision required for acceptable performance.
13 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
Results across this thread
14 reported results from the papers in this thread.
The table is for subscribers. Subscribe to see every reported number side by side.
How this thread developed
Oct 2026 · Mutah University
Neural network precision floor follows a depth law predicted by amplification
Derives a depth law for the precision floor of neural networks, relating network depth to the minimum bit-width for acceptable accuracy.
released code
Oct 2026
Tetra serves 2.7-bit language models without expanding lattice codes
Demonstrates practical serving at 2.7 bits using Leech-lattice quantization, advancing understanding of minimum precision feasibility.
1 further paper
Oct 2026 · Carnegie Mellon University, ML Collective
Compressed looped models settle, not drift, so precise final loops recover them
Reveals that compressed looped models settle to fixed error, enabling recovery with precise final loops.
1 further paper
Oct 2026 · KAIST, Samsung AI Center-Cambridge
Recalibrating activations helps quantized vision transformers withstand distribution shifts
Identifies a quantization-specific failure mode under distribution shift and proposes activation recalibration, adding a robustness dimension to precision scaling.
1 further paper
Oct 2026 · Meta AI
Canonical training improves language models under joint low-bit compression
This paper introduces CanonQ, a quantization-aware training framework for joint low-bit compression (W2A4KV2), advancing precision scaling for multiple model parts.
4 further papers
Oct 2026 · Alibaba Token Hub, Alibaba Group, Ohio State University
Aligning FP4 training and rollout stabilizes reinforcement learning
aligns FP4 training and rollout to maintain RL stability under low precision
6 of 13 papers shown