Many-Tier Instruction Hierarchy in LLM Agents

Researchers propose Many-Tier Instruction Hierarchy (ManyIH), a new framework for resolving conflicts among instructions given to large language model agents from multiple sources with varying authority levels. Current models achieve only ~40% accuracy when navigating up to 12 conflicting instruction tiers, revealing a critical safety gap in agentic AI systems.

The paper addresses a fundamental vulnerability in deployed AI agents: their inability to reliably prioritize instructions when conflicts arise from multiple sources. As LLM agents become more autonomous and integrated into complex workflows, they receive directives from system prompts, users, tool outputs, and external APIs—each with different legitimacy levels. Traditional instruction hierarchy frameworks assume fewer than five privilege tiers based on rigid categories like 'system > user,' an oversimplification that breaks down in real-world environments where instruction sources proliferate.

This research emerges as AI safety becomes increasingly sophisticated. Previous work focused on single-source jailbreaks and prompt injection; ManyIH-Bench escalates the challenge by introducing realistic multi-source conflict scenarios across 46 real-world agents. The benchmark's composition—427 coding tasks and 426 instruction-following tasks spanning 12 privilege levels—reflects genuine deployment complexity. Human verification of test cases ensures validity beyond synthetic adversarial examples.

The performance gap matters significantly for production systems. At ~40% accuracy on frontier models, instruction conflict resolution remains unreliable enough to threaten both safety (following low-privilege malicious instructions) and functionality (rejecting legitimate high-privilege directives). Organizations deploying autonomous agents cannot tolerate this failure rate in security-critical contexts like financial transactions, data access, or system administration.

Looking forward, this work establishes a new benchmark category similar to how HELM advanced language model evaluation. Development teams must either enhance existing models' reasoning about privilege hierarchies or implement external verification layers. The research signals that instruction safety scaling requires explicit, deliberate architectural solutions rather than emergent capabilities from scale alone.

• →Current frontier LLM agents fail on ~60% of multi-tier instruction conflict scenarios, revealing a critical safety vulnerability
• →ManyIH-Bench introduces the first standardized evaluation with up to 12 privilege levels across 46 real-world agents
• →Traditional instruction hierarchy frameworks with fewer than 5 tiers are inadequate for complex, real-world agentic deployments
• →The research indicates that instruction safety scaling requires explicit architectural solutions beyond model scale increases
• →Production AI systems face significant reliability and security risks until fine-grained conflict resolution methods improve