The paper investigates whether task vectors produced by (OPD) can be merged effectively with those from reinforcement learning (RL) teachers. Across five domains and two model architectures, the authors find that within a task, merging a weaker OPD student with its RL teacher produces a model that outperforms either constituent. Across tasks, compositions of OPD task vectors achieve higher average scores than corresponding RL compositions in seven of eight comparisons.
These findings are interesting because they show that a model with lower standalone performance can still contribute useful parameter updates when merged, and that OPD updates may interfere less when combining multiple domain experts. The results suggest that composability is a distinct property of post-training updates, separate from the usual focus on single-model performance.