As AI agents are used in coding, banking, and commerce, they can be tricked into violating security rules through hidden instructions in content they process. Testing their defenses requires realistic environments, but existing often test the agents themselves rather than the tools that generate attacks.
GreyZone is a benchmark built around ten real-world agent applications, each with a security policy. It measures how well automated red-teaming services, programs that automatically probe for weaknesses, can deliver attacks that actually break those policies. Each scenario has a verified outcome check, so a failure means the service missed a real vulnerability, not that the system is secure.
In two case studies using the Promptfoo red-teaming service, GreyZone showed that services can adapt their attacks based on system feedback, turning initial refusals into eventual successes. This highlights the need to evaluate red-teaming services over full attack sequences, not just single attempts.