Joint RL and self-distillation trains agents on task rewards plus dense teacher guidance, but the two gradient signals can partially cancel.
Projecting conflicting gradients during joint RL and distillation improves agent training
CORD selectively removes opposing gradient components while preserving a protected objective, boosting task success on ALFWorld and WebShop across several training recipes.
Chinese Tech
Wenhao Yu · Zhengcheng Wang · Jiahong Liu · Zexuan Qiu · Minda Hu · Shaohang Wei · +4 more
The Chinese University of Hong Kong · Tencent · Peking University
Research Digest··3 min read
The authors introduce CORD, a conflict-aware gradient combination framework for joint reinforcement learning and self-distillation.
§