Two-memory approach lets AI agents learn new tasks without forgetting old ones

The authors show that splitting a reinforcement learning agent's knowledge into a permanent, parametric system and a transient, non-parametric system improves performance on image-based and procedural tasks.

Big Tech

Nishanth Anand · Doina Precup

McGill University · Mila · Google DeepMind

Research Digest·6 November 2026·2 min read

Illustration: Two-memory approach lets AI agents learn new tasks without forgetting old ones

In one line

Separating value functions into permanent parametric and transient non-parametric features balances stability and plasticity in continual reinforcement learning.

Before you read this

Three ideas this paper builds on

continual learningA learning paradigm where a model sequentially acquires knowledge from multiple tasks without significantly forgetting earlier ones.59 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.272 papers →memory consolidationSummarizing or structuring past information to reduce token usage while preserving task accuracy.38 papers →

Read it as

agents struggle to adapt to new situations without forgetting old ones, a problem called the stability-plasticity tradeoff.

This paper extends a brain-inspired dual-system approach by using separate features for each. A neural network learns slow, generalizing permanent knowledge, while a novel non-parametric approximator based on MinHash captures fast, context-specific transient corrections.

The resulting agent learns online and outperforms strong baselines on image-based tasks and Craftax. The work shows that distinct permanent and transient feature representations are essential for balancing stability with plasticity, offering a concrete path to robust continual learning.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

continual learning reinforcement learning complementary learning systems memory consolidation