The authors distinguish goals, which specify what to accomplish, from constraints, which restrict how to accomplish it.
Training on competing goals improves models’ handling of instruction conflicts
Chen and colleagues report that synthetic text-processing conflicts improve instruction-priority adherence in two models, with little average change on tasks without conflicts.
Big Tech
Sizhe Chen · Sanjay Kariyappa · Liwei Jiang · Chong Xiang · Jihye Choi · Chaowei Xiao · +1 more
NVIDIA · UC Berkeley
Research Digest··2 min read
Language models should follow user requests only when they do not conflict with higher-priority system instructions.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.