TJPO is a method for RL-based post-training of LLM agents that explicitly encourages behavioral diversity in trajectories. It allows users to define task-specific descriptors (e.g., action types, locations visited) and measures diversity across sampled trajectories. This diversity is then used as a learning signal to reward varied successful strategies.
Experiments on Sokoban and ALFWorld show TJPO improves diversity metrics while maintaining competitive task success. It also provides practical benefits: on ALFWorld, TJPO improved compliant success under user-specified location constraints by 8.3 percentage points and task success under environmental disruptions by 7.5 percentage points compared to a baseline.
This work is interesting because it makes diversity interpretable and controllable, directly targeting the behavioral variation that matters for a given task, rather than relying on implicit stochasticity. This can help agents satisfy user preferences and adapt to changing conditions.