Transformer context tokens can be absorbed into the model's weights as a multiplicative operator. For a safety instruction, this operator is a rank-1 perturbation of the identity, so it has a single dominant direction with an associated safety eigenvalue. The proposed Contrastive Safety Loss uses a suppression weight that amplifies the eigenvalue on harmful queries and suppresses it on harmless ones.
Varying the suppression weight traces a continuous tradeoff between attack success rate and over-refusal rate. The relationship holds across different parameterizations of the loss, and appropriate suppression values yield Pareto-improved safety instructions. This is interesting because it turns safety prompt tuning into a spectral optimization problem with a single continuous dial.