The authors used the difference between a model’s token probabilities before and after safety fine-tuning to construct an amplification direction.
Reversing safety fine-tuning helps estimate failures too rare to count
ALE amplifies suppressed failures, then uses token probabilities to estimate their original frequency, reaching rates near one in 100,000 in Qwen3-8B experiments.
Research Lab
Patrick Reichherzer · Jonathan Evertz · Thorsten Holz
Max Planck Institute for Security and Privacy (MPI-SP) · CISPA Helmholtz Center for Information Security
Research Digest··2 min read
Reichherzer and colleagues introduce Amplify, Locate, Estimate (ALE), a method that makes a fine-tuned model’s rare failures more common so they can be measured cheaply, then estimates their frequency in the original model.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.