The authors discretize possible returns into bins and train a return-conditioned policy to model the action likelihood p(a|s,z), where s is the state and z is a candidate return.
Conditioning actions on returns improves value learning in reinforcement learning
The authors build a critic from a return-conditioned generative policy, then anchor its value estimates to standard temporal-difference targets.
Top University
Alexander Swerdlow · Perry Dong · Chelsea Finn
Stanford University
Research Digest··2 min read
Swerdlow, Dong and Finn reverse the usual value-learning problem: instead of predicting returns directly from state-action pairs, their Generative Value Classifier models how likely an action is under each possible return.
§