The paper introduces StepLearn, a framework that allows LLM agents to learn during deployment without retraining model parameters. It addresses the problem that existing test-time learning methods only update knowledge after completing an episode, which delays the use of new information. StepLearn instead learns from individual transitions, immediately using potentially useful rules for the next decision but only promoting them to persistent memory after they have been prospectively validated in a different episode. This balances immediate usefulness with reliability. Evaluated on WebArena-Lite and ALFWorld, StepLearn outperformed the best baseline by 2.2-12.7 percentage points, and the authors note that gains are present even on first task attempts, not just after repeated runs.
StepLearn lets agents use new knowledge immediately while requiring validation before persistent reuse.
The nonparametric framework improves LLM agent success on WebArena-Lite and ALFWorld by separating immediate guidance from cross-episode trust.
Chinese Tech
Tong Zhao · Reed Li · Yuyang Hu · Yutao Zhu · Haijin Liang · Haibo Shi · +2 more
Renmin University of China · Tencent
Research Digest··3 min read
In one line
StepLearn lets LLM agents immediately use insights from ongoing interactions while requiring cross-episode validation before long-term adoption.
Before you read this
Three ideas this paper builds ontest-time adaptationAdapting a deployed model during inference using new data or interactions, without a separate training phase.84 papers →agent memoryA repository where an agent stores past interactions and outcomes to inform future decisions.241 papers →verificationChecking outputs for correctness before accepting them, often to catch errors or reward hacking.341 papers →
Read it as
The authors introduce StepLearn, a nonparametric framework for test-time learning in LLM agents.
StepLearn is a prequential, nonparametric learning framework.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.