PARTS is a real-world reinforcement learning framework for adapting a frozen pretrained robot policy to long-horizon tasks. Instead of collecting new full-task demonstrations or relying only on sparse full-task rewards, it identifies bottleneck subtasks and trains small residual policies that correct the frozen base policy only within those windows. Selectors decide when residual corrections activate, and success verifiers provide local outcome rewards, so learning can make progress even when the complete task has not yet been solved.
Training combines online RL with success-reweighted retraining over accumulated experience, and each retrained residual policy is deployed to gather more rollouts. Humans only identify bottlenecks during setup and perform physical resets when needed. On two bimanual YAM tasks and one single-arm Franka task, full-task success rose from 32% to 61% and from 50% to 95%, with tens of minutes of real-world rollouts per task on average. PARTS also raised full-task success by more than 25% compared with existing real-world RL fine-tuning methods under the same rollout budget while needing less human involvement.