The authors developed NoteBank, a training-free memory interface for recurrent and hybrid language models.
Sparse replay memory extends linear attention to million-token contexts
NoteBank stores selected text spans separately and replays retrieved evidence through pretrained recurrent models without retraining.
Top University
Li Nanbo · Firas Laakom · Yilan Zhang · Piotr Piękos · Yimeng Chen · Liangyu Wang · +7 more
King Abdullah University of Science and Technology (KAUST)
Research Digest··2 min read
Li Nanbo and colleagues augment linear-attention language models with a sparse episodic memory that preserves selected text outside the model’s fixed recurrent state.
Why this paper
From King Abdullah University of Science and Technology (KAUST)
In one line
NoteBank, a training-free episodic memory, improves long-context retrieval in linear attention from 1.7% to 70.3% at one million tokens.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§