Large language models (LLMs) are AI systems that generate text. They are often trained to refuse harmful requests, but that refusal behavior can be lost after additional fine-tuning, which is further training on new data. This is a problem because models need to stay safe as they are updated.
The authors found that the typical refusal responses, short, formulaic phrases like “Sorry, I cannot help”, are easily learned but easily overwritten by later fine-tuning. They proposed NEST, a method that constructs more natural, context-aware safety responses that fit the model’s existing language patterns.
Using NEST, safety behaviors were retained much better after additional training while the model remained helpful. This suggests that durable depends on designing safety targets that are natural and semantically grounded in the request.