The authors prove that sending a Gaussian vector through a layer’s quantization-error matrix gives an unbiased estimate of the error’s squared Frobenius norm, a measure of total rounding error.
One random probe can guide data-free mixed-precision model quantization
The authors show that spectrally flat rounding errors make Gaussian probes reliable enough to allocate per-tensor bit widths under an exact memory budget.
Industry
I Kennedy · T Kennedy
Blacksheep ai.
Research Digest··2 min read
Kennedy and Kennedy develop RAM, a post-training quantization method that scores model tensors without a calibration dataset.
Why this paper
From Blacksheep ai.
In one line
Round-to-nearest quantization error is spectrally flat, letting one Gaussian probe estimate tensor sensitivity and allocate mixed precision without calibration data.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (params 400B)
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (10 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§