The paper introduces S3Gym, a to test whether LLMs can improve through their own experience. It uses seven text-based games where agents explore freely then are evaluated on held-out configurations. Three pathways for using experience are tested: feeding raw history in context, using compressed summaries, or updating model parameters.
Results show that self-improvement is not automatic. Summaries help when rules can be abstracted, but raw history works better for tasks needing precise state information. Parameter training sometimes gives large gains but also causes instability and negative transfer on other tasks.
This matters because it shows recognizing success is not enough; agents must turn feedback into usable policies. S3Gym provides a way to diagnose where self-improvement breaks down, helping researchers choose the right pathway for their task.