When you use a chatbot or tool that chains together multiple safety checks and approvals, the order those checks run matters. This paper studies systems called LLM , which coordinate AI models, tools, and safety rules. A problem arises when everyday tasks, like reloading a plugin or saving a setting, change the order in which checks happen, even though no code was changed.
The researchers examined five common safety patterns across five real systems and fourteen plugins. They traced how each safety requirement was supposed to work, then tested what happened during routine operations like reapplying a plugin or saving an allow rule. They found that after such operations, some safety checks were silently skipped, allowing prohibited actions to proceed.
One key finding: after a switch from observe mode to enforce mode, a policy plugin was moved behind an approval plugin, so an approved prohibited edit ran without a policy check. This matters because system architects now know that a pattern's safety guarantee is not fixed; it depends on the history of operations applied to the system.