What they did
The authors encoded the same 48 synthetic histories in four memory formats: verbatim long-context histories (LC-RAW), chunked retrieval-augmented generation stores (RAG), model-compressed natural-language notes (NOTES), and fixed-schema knowledge graphs (KG-fixed). Randomized answer codes and exact-match scoring were used to reduce contamination and scoring ambiguity.
They then migrated memory between two open-weight models with fewer than 10 billion parameters, varying which model constructed and consumed the memory. Additional experiments separated information loss from retrieval failure, compared full and partial embedding migrations, and tested whether memories could be repaired with or without the raw source histories.
Key findings
- KG-fixed was nearly invariant to a writer-model swap: accuracy changed by just +0.0004 ± 0.0020.
- NOTES migrations were asymmetric: accuracy changed by +9.91 percentage points in one direction but −13.28 points in the other, indicating strong coupling between the model that wrote the notes and the model that later interpreted them.
- A 50/50 mixed-embedding RAG index improved accuracy by 4.96 points, compared with an 11.90-point gain from fully re-embedding the store. Retrieval failures accounted for 81% of the measured RAG deficit, or 0.364 ± 0.012.
- Information discarded when NOTES were first constructed accounted for 80% of their accuracy deficit, or 0.467 ± 0.014. Store-only repair missed a 90% recovery target in all 48 cases; retaining raw histories enabled recovery in 34 of 48 cases in one migration direction.
Why it matters
The results suggest that agent memory should be treated as a versioned system component rather than a model-independent archive. Fixed schemas, isolated embedding spaces, direction-specific migration tests, and retained source histories can make model upgrades less likely to silently alter what an agent recalls.
Caveats
The study covers only 48 synthetic histories and two open-weight models below 10 billion parameters, using randomized codes and exact scoring rather than realistic conversational or workplace tasks. It remains unclear whether the same effect sizes hold for larger models, other embedding families, evolving schemas, noisy real-world histories, or repeated migrations over time.