CaMeLs Can Use Computers Too: System-level Security for Computer Use Agents

Researchers introduce NOVA, a security architecture for Computer Use Agents that prevents prompt injection attacks through upfront branching plans and architectural isolation. The system maintains up to 57% performance parity with frontier models while improving smaller models by 19%, though new vulnerabilities like Branch Steering attacks remain.

The research addresses a critical vulnerability in AI agents designed to automate computer tasks by viewing screens and executing actions. Prompt injection attacks represent a significant security threat where malicious content can hijack agent behavior, necessitating robust defenses. The NOVA framework resolves a fundamental tension between security and functionality: while isolation provides strong guarantees against instruction injection, Computer Use Agents require continuous UI observation to determine appropriate actions.

This work builds on growing recognition that AI agent security demands architectural rather than purely behavioral solutions. The insight that UI workflows, despite their dynamic appearance, follow predictable structural patterns enables single-shot planning—where agents emit complete branching plans upfront rather than making decisions reactively. By allowing perception models to resolve runtime values like UI coordinates within a pre-established control flow, the system maintains integrity against arbitrary instruction injections.

The evaluation on OSWorld demonstrates practical viability: frontier models retain substantial performance while smaller open-source models see performance improvements, suggesting security doesn't necessitate capability trade-offs. This matters for developers building automated systems that must operate in security-sensitive environments like finance or healthcare.

However, the identification of Branch Steering attacks—where adversaries manipulate perception models into routing execution through attacker-preferred plan branches—reveals that architectural isolation alone proves insufficient. This emerging attack class requires perception-layer defenses beyond the current framework. The research trajectory indicates that robust AI agent security requires defense-in-depth strategies combining architectural isolation, verified planning, and robust perception mechanisms.

• →NOVA framework prevents prompt injection attacks in Computer Use Agents through architectural isolation and upfront control flow planning
• →Single-shot planning with runtime perception resolution enables security without severely compromising agent performance
• →Smaller open-source models improve performance by up to 19% under the NOVA architecture, indicating security benefits extend beyond frontier models
• →Branch Steering attacks demonstrate that architectural isolation alone cannot defend against adversarial manipulation of perception models
• →The research suggests defense-in-depth approaches combining planning, isolation, and perception robustness are necessary for secure AI agents