MemProbe treats like cognitive memory: controlled input sequences are used to observe how a system updates, preserves, attributes, and temporally organizes facts. Four paradigms, interference, misinformation, consolidation strength, and reconsolidation window, each perturb memory in a specific way and probe the expected behavioral signature.\n\nThe framework was instantiated as a 56-episode diagnostic suite and run on six incremental memory systems under a unified protocol. Systems with similar aggregate scores showed distinct behavioral profiles, meaning the same final accuracy can hide very different update behavior.\n\nThis matters because final-answer accuracy alone cannot tell whether a system over-writes, retains outdated facts, or handles conflicting evidence in a risky way. MemProbe turns that single score into a multi-dimensional stability-plasticity profile and a failure taxonomy.
Memory benchmarks can hide how agents preserve and revise facts
MemProbe uses cognitive experiments to reveal whether agent memory systems overwrite, retain, attribute and order changing information appropriately.
Industry
UNC-Chapel Hill
Research Digest··2 min read
In one line
Aggregate accuracy hides how agent memory systems update, preserve, attribute, and temporally organize facts; four cognitive paradigms expose distinct stability-plasticity profiles.
Before you read this
Three ideas this paper builds onagent memoryA repository where an agent stores past interactions and outcomes to inform future decisions.241 papers →agentic evaluationMeasuring AI agents' performance in complex tasks, highlighting failures in planning, monitoring, and real304 papers →memory consolidationSummarizing or structuring past information to reduce token usage while preserving task accuracy.40 papers →
Read it as
Ding and Wu introduce MemProbe, a framework for evaluating the stability-plasticity balance between preserving existing memories and incorporating new evidence.
The authors translated four cognitive-memory paradigms into controlled agent evaluations: interference, misinformation, consolidation strength and the reconsolidation window.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.
What it released
Code