The authors tested eight jailbreak methods on 50 AdvBench requests across three open-weight models.
Jailbreak success rates flip with evaluator and sampling choices
Comparing eight attacks on three open-weight models, the authors show that the same attacks rank first or last depending on evaluator, and that repeated sampling can surface unsafe responses after a refusal.
Academic
Leon Tu Nguyen · Rohan Menon · Rafael Tappe Maestro · Eric Aubinais · Chih-Hong Cheng
University of Oldenburg · Chalmers University of Technology
Research Digest··2 min read
The authors compared eight jailbreak attacks on Llama 2-7B, Vicuna-7B, and Qwen3-8B using three evaluation procedures.
§