instruction hierarchy
- Papers
- 4
- Released code
- 0
- First seen
- Aug 2026
- Latest
- Nov 2026
3 papers in the last two months, against 1 in the two before.
Start here
Two papers to read first, in the order they came out- 1Agent harnesses can elevate malicious context past instruction defenses
Instruction privilege escalation re-labels malicious tool-level content as user or system instructions, yielding attacks across six coding agents and bypassing permission review.
Aug 2026 - 2Chatbots follow higher-priority rules better after training on conflicting goals
Training on goal-goal conflicts significantly improves language models' adherence to instruction hierarchy.
Oct 2026
How fast it is moving
Often studied with
synthetic data generationCreating training examples artificially, often using an LLM, to overcome real-data scarcity or privacy constraints.2 togethermulti-agent architectureA system where multiple AI agents collaborate, each handling specialized roles, communicating and coordinating to complete complex tasks.2 togetherprompt injectionA security attack where malicious instructions are embedded into inputs to override an LLM's intended behavior.2 togetherThe papers
Most central to this idea first, not most recent.
- Big Techcs.AI
Chatbots follow higher-priority rules better after training on conflicting goals
Training on goal-goal conflicts significantly improves language models' adherence to instruction hierarchy.
NVIDIA, UC Berkeley · Oct 2026
- Top Universitycs.CR
Agent harnesses can elevate malicious context past instruction defenses
Instruction privilege escalation re-labels malicious tool-level content as user or system instructions, yielding attacks across six coding agents and bypassing permission review.
Nanjing University, Honor Device Co., Ltd · Aug 2026
- Big Techcs.CR
AI agents remain vulnerable to prompt injection when attackers use collaborating swarms
Prompt injection attacks remain effective against frontier AI agents when attackers use swarms of collaborating LLMs.
Google DeepMind · Nov 2026
- Big Techcs.CR
Fake content and structure in tool outputs combine to fool all tested AI agents
Role spoofing attacks that mimic user style and format fool LLMs over 95% of the time.
University of Pennsylvania, Princeton University · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.