What they did

The authors built an interactive benchmark around seven text-based games with executable environment verifiers. Its protocol evaluates three linked capabilities: generating exploratory experience through self-testing, assessing that experience through self-judging, and using it to improve performance on a separate held-out evaluation.

They compared three ways of incorporating experience: History ICL, which places prior interactions directly in the model's context; score-conditioned Summary Memory, which compresses experience into reusable guidance; and parameter Training, which updates the model itself.

Key findings

• Context-level experience improved performance for several model–game combinations, but improvement was not consistent across the benchmark.
• Summary memories worked best when past experience could be distilled into general strategic rules.
• Raw interaction histories often outperformed summaries when decisions depended on exact, state-specific information that compression could discard.
• Parameter training produced substantial gains on some tasks but was unstable and caused severe negative transfer—learning that reduced performance—on others.

Why it matters

S³Gym shifts evaluation from asking whether an agent performs well once to asking whether it can convert accumulated interactions into better future behavior. The results challenge the assumption that recognizing successful actions is enough: reliable self-improvement also requires converting feedback into policies that remain executable and transfer to new situations.

Caveats

The evidence comes from seven text-based games, so it is unclear how the findings extend to richer software, web, or embodied environments. The abstract does not report aggregate effect sizes or establish a universally superior learning pathway; instead, it emphasizes strong dependence on the particular model and task.