A new fine-tuning method treats safety as a reliability constraint rather than an average penalty. Instead of minimizing average safety loss, the method limits the fraction of safety examples whose performance degrades beyond a set threshold. The approach uses a differentiable approximation of the chance constraint and a gradient correction that minimally modifies the fine-tuning direction to stay within a safe region.
Experiments on three (Qwen 3.5 4B, 9B, and Llama 3.1 8B) across three poisoned tasks showed consistent improvements over existing defenses. The method achieved better fine-tuning accuracy while reducing harmful regressions.
This work reframes safety preservation as a constrained optimization problem, offering a principled way to control tail risks that average-based methods miss.