The authors propose the Continuous Memory Machine (CMM), building on the Continuous Thought Machine (CTM) which models short-term neural dynamics.
Separate short- and long-term matrix memories improve recurrent network performance
The authors' Continuous Memory Machine uses a shared Transformer to update both memory stores, outperforming baselines on algorithmic, in-context learning, and reasoning tasks.
AI Startup
Ciaran Regan · Kai Arulkumaran · Luke Darlow · Stefania Druga · Sebastian Risi · Llion Jones
University of Tsukuba · Sakana AI
Research Digest··2 min read
Regan et al.
Why this paper
From Sakana AI and University of Tsukuba · Released code
In one line
Continuous Memory Machines outperform prior recurrent networks by separating short- and long-term memory into matrix-valued states.
What it released
Code
What we could check
- ✓Code link in the paper (github.com)
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§