AI red-teaming is a process where teams deliberately try to make a generative AI system produce harmful content, such as dangerous advice or biased statements, to find weaknesses before the system is used. This testing is often seen as a way to measure safety.
The researchers studied whether the specific wording of the instructions given to red-teamers changes what they discover. They reviewed how instructions were written in past projects and then ran controlled tests where only the instructions varied, while everything else stayed the same. Nineteen thousand simulated attacks were performed across five different AI models.
The main finding was that instruction length and detail strongly affect the number of successful attacks found. Instructions that were very short or very long found more harms than medium-length instructions, and including examples increased success, while defining harm in detail reduced it. This shows that human-written instructions, not just the AI model, shape safety evaluations.