VaDar decomposes a long-horizon task into discrete subgoals, then trains a subgoal-independent policy using a critic that simultaneously estimates the value of each subgoal.
Value Disaggregation trains policies without inference-time goal conditioning
VaDar uses a single subgoal-agnostic policy with a critic that estimates values of all subgoals, eliminating the need for subgoal selection during deployment.
Big Tech
Shashwat Saxena · Mehul Goel · Sreyas Venkataraman · Sarvesh Patil · Max Simchowitz
Carnegie Mellon University · Amazon
Research Digest··2 min read
The authors introduce Value Disaggregation (VaDar), a reinforcement learning framework that trains a policy to maximize a linear combination of per-subgoal critic values.
§