Safety Training Side Effects
Investigating how safety training can inadvertently transform harmful behaviors (e.g., discrimination) into subtler forms rather than eliminating them, and how to detect or mitigate these side effects.
5 papers
Where this stands
The written synthesis of this thread is for subscribers. Subscribe.
How this thread developed
Sept 2026 · Durham University
Safety training can launder gender discrimination into subtler forms, study finds
The study demonstrates safety training transforms overt discrimination into subtler forms, a core example of side effects in safety training.
1 further paper
Sept 2026 · Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences
Post-training alignment makes language models more confidently wrong
Documents a new side effect where alignment training increases confidence in wrong long-tail answers and traces its emergence across layers.
1 further paper
Oct 2026 · University of Southern California, Carnegie Mellon University
Training on aligned data can cause misalignment in different contexts
Shows that alignment fine-tuning in one context can cause misalignment in other contexts, a clear safety training side effect.
released code
3 of 5 papers shown