AD-Memo lets a vision-language-action driving agent periodically describe critical surrounding objects in text, then feeds that text back as context at future steps. This offers a cheaper, human-readable alternative to retaining long sequences of image tokens or opaque latent memories.
Training combines supervised fine-tuning with Da Capo, a semi-closed-loop reinforcement-learning method that uses the agent’s own prior memories while replaying ground-truth driving states. Evaluations on all-way stops and general driving report better driving quality and scene question answering, while the generated memories can also be consumed by other models. The provided text does not include quantitative results.