instruction hierarchy
- Papers
- 3
- Released code
- 0
- First seen
- Aug 2026
- Latest
- Oct 2026
2 papers in the last two months, against 1 in the two before.
Start here
Two papers to read first, in the order they came out- 1Agent harnesses can elevate malicious context past instruction defenses
Instruction privilege escalation re-labels malicious tool-level content as user or system instructions, yielding attacks across six coding agents and bypassing permission review.
Aug 2026 - 2Chatbots follow higher-priority rules better after training on conflicting goals
Training on goal-goal conflicts significantly improves language models' adherence to instruction hierarchy.
Oct 2026
How fast it is moving
Often studied with
agent harnessA system that structures, verifies, monitors, or controls how an AI agent interacts with external tools, environments, or execution runs.1 togethermulti-agent architectureA system where multiple AI agents collaborate, each handling specialized roles, communicating and coordinating to complete complex tasks.1 togetherpreference optimizationA post-training method that adjusts model behavior using comparisons or self-generated preferences to improve alignment.1 togetherThe papers
Most central to this idea first, not most recent.
- Big Techcs.AI
Chatbots follow higher-priority rules better after training on conflicting goals
Training on goal-goal conflicts significantly improves language models' adherence to instruction hierarchy.
NVIDIA, UC Berkeley · Oct 2026
- Top Universitycs.CR
Agent harnesses can elevate malicious context past instruction defenses
Instruction privilege escalation re-labels malicious tool-level content as user or system instructions, yielding attacks across six coding agents and bypassing permission review.
Nanjing University, Honor Device Co., Ltd · Aug 2026
- Big Techcs.CR
Fake content and structure in tool outputs combine to fool all tested AI agents
Role spoofing attacks that mimic user style and format fool LLMs over 95% of the time.
University of Pennsylvania, Princeton University · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.