The authors introduce LMOPD for integrating reward-specialized policies under a lexicographic priority order.
Lexicographic distillation preserves priority-ordered capabilities in multi-objective RL
Method routes student rollouts to highest-priority unmet specialist and projects corrections to avoid interfering with higher-priority experts.
AI Startup
Doseok Jang · Jon Ander Campos · Youran Qi
Cohere · Mila, Université de Montréal
Research Digest··3 min read
The authors propose LMOPD, a multi-teacher distillation method that uses a lexicographic priority order to route each student rollout to the specialist for the highest-priority unmet objective.
Why this paper
From Cohere and Mila, Université de Montréal · Part of Reasoning Distillation Alignment, now 14 papers
In one line
LMOPD preserves top-priority capabilities by routing each rollout to the specialist for its highest unmet objective and projecting corrections.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (params 30B-A3B)
- ✓Limitations stated by the authors (2 noted)
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§