Li et al.
Two-sided associative memory correction boosts linear attention performance
Gated Slot Attention-2 uses Oja and delta rules to correct both key and value sides of recurrent memory, showing gains across multiple benchmarks.
Chinese Tech
Ruijie Li · Shengnan Ding · Weimin Zhang · Derick Tang · Zhanpeng Zeng · Qinsong Zeng · +3 more
The Hong Kong University of Science and Technology (Guangzhou) · Tencent
Research Digest··2 min read
The authors introduce Gated Slot Attention-2 (GSA2), which combines Gated Oja Rule-2 for key-side correction and Gated Delta Rule-2 for value-side correction via shared latent slots.
Why this paper
From Tencent and The Hong Kong University of Science and Technology (Guangzhou)
In one line
Gated Slot Attention-2 corrects both key and value memory in linear attention using complementary update rules.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§