The authors build on formulations by Chen and Li (2023) and Bui et al.
Random Fourier features make Gaussian process attention linear-time and well calibrated
The proposed RFF-GPA module approximates attention as a Gaussian process posterior with a random Fourier feature kernel, cutting sequence-length complexity to linear while improving calibration on classification benchmarks.
Independent
Amir Mohammad Mahfoozi · Zi Yang · Ying Li · Michael Minyi Zhang
Research Digest··2 min read
The authors propose RFF-GPA, a plug-and-play Transformer attention module that treats attention as a Gaussian process posterior with a kernel approximated by random Fourier features.
Why this paper
Independent
In one line
Random Fourier feature Gaussian process attention provides linear-time complexity and calibrated uncertainty for transformer models.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§