3B) models during autoregressive decoding.
Recurrent-state quantization fails at long decode horizons due to scale-state feedback
The authors show that per-token dynamic INT4 quantization causes norm explosion and perplexity collapse after hundreds of steps, and that containing the scale rather than locating outliers is the correct mitigation.
Independent
Yuhang Jiang
Independent Researcher
Research Digest··3 min read
The authors study training-free INT4 quantization of the recurrent state in linear-attention (Gated DeltaNet) and state-space (Mamba-2) language models.
Why this paper
From Independent Researcher
In one line
Per-token dynamic scaling of recurrent state quantization fails over long decode due to scale-state feedback; persistent scaling outperforms.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§