The authors designed a delta-style rule that corrects a scalar key-value association by changing the memory matrix along a single direction.
Targeted memory updates improve stable linear attention over long contexts
KronLSTM revises individual associations along one matrix direction, limiting interference while keeping its recurrent memory bounded.
Independent
Anamaria-Roberta Hartl · Levente Zólyomi · Michael List · Günter Klambauer · Sebastian Böck · Sepp Hochreiter
Research Digest··3 min read
Hartl and colleagues introduce the Kronecker Delta Rule, an update mechanism for associative matrix memories, and build it into a linear-attention architecture called KronLSTM.
§