← All threads

Precision Scaling Laws

Investigating the relationship between network depth, parameter perturbations, and the minimum precision required for acceptable performance.

13 papers

Where this stands

The written synthesis of this thread is for subscribers. Subscribe.

Results across this thread

14 reported results from the papers in this thread.

PaperBenchmarkMetricResultConditionsCode

The table is for subscribers. Subscribe to see every reported number side by side.

How this thread developed

  1. Oct 2026 · Mutah University

    Neural network precision floor follows a depth law predicted by amplification

    Derives a depth law for the precision floor of neural networks, relating network depth to the minimum bit-width for acceptable accuracy.

    released code

  2. Oct 2026

    Tetra serves 2.7-bit language models without expanding lattice codes

    Demonstrates practical serving at 2.7 bits using Leech-lattice quantization, advancing understanding of minimum precision feasibility.

  3. 1 further paper

    Oct 2026 · Carnegie Mellon University, ML Collective

    Compressed looped models settle, not drift, so precise final loops recover them

    Reveals that compressed looped models settle to fixed error, enabling recovery with precise final loops.

  4. 1 further paper

    Oct 2026 · KAIST, Samsung AI Center-Cambridge

    Recalibrating activations helps quantized vision transformers withstand distribution shifts

    Identifies a quantization-specific failure mode under distribution shift and proposes activation recalibration, adding a robustness dimension to precision scaling.

  5. 1 further paper

    Oct 2026 · Meta AI

    Canonical training improves language models under joint low-bit compression

    This paper introduces CanonQ, a quantization-aware training framework for joint low-bit compression (W2A4KV2), advancing precision scaling for multiple model parts.

  6. 4 further papers

    Oct 2026 · Alibaba Token Hub, Alibaba Group, Ohio State University

    Aligning FP4 training and rollout stabilizes reinforcement learning

    aligns FP4 training and rollout to maintain RL stability under low precision

6 of 13 papers shown