Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?

Researchers introduced AgentCIBench, a safety testing framework that reveals critical privacy vulnerabilities in computer-use agents (CUAs) that access multiple personal applications. Testing 15 frontier agents found that 11 leak sensitive information on over 50% of scenarios, exposing risks from UI co-location, task ambiguity, and recipient misalignment.

Computer-use agents represent a significant shift in how users interact with digital systems, automating tasks across email, calendars, and personal applications simultaneously. However, this convenience creates a substantial privacy liability that the field has largely ignored: agents lack contextual awareness and often pull information from one context into another where it's inappropriate. The AgentCIBench study systematically quantifies this risk through three failure modes—visual co-location where agents grab adjacent UI elements, task-ambiguity overshare where vague prompts trigger excessive data disclosure, and recipient misalignment where sensitive content reaches unintended recipients.

The empirical findings are stark. With 67.9% average information leakage across 15 tested agents and 11 of 15 exceeding 50% failure rates, the research suggests that current commercial agents prioritize capability over privacy. This pattern persists even in end-to-end task execution, indicating the problem is fundamental rather than superficial.

The implications extend across multiple stakeholders. For developers building agent systems, the benchmark provides concrete failure scenarios that demand architectural solutions. For enterprise adoption, these leakage rates present significant compliance and liability risks, particularly in regulated industries handling sensitive data. For users, the findings highlight that delegating control to agents requires understanding their privacy guarantees—or lack thereof.

The release of AgentCIBench as an open evaluation harness signals that contextual integrity testing will likely become a pre-deployment requirement. This mirrors similar standardization efforts in AI safety and could reshape how agents are designed, trained, and deployed.

• →11 of 15 frontier agents leaked sensitive information on more than 50% of test scenarios, averaging 67.9% failure rate
• →Three primary failure modes identified: visual co-location, task-ambiguity overshare, and recipient misalignment
• →Privacy vulnerabilities persist in end-to-end agent execution, indicating the issue is structural rather than situational
• →AgentCIBench provides a deterministically scored evaluation harness to measure contextual integrity in agent systems
• →Contextual disclosure testing is emerging as a critical pre-deployment safety check for computer-use agents