SkillHarness: Harnessing Safe Skills for Computer-Use Agents

Researchers introduce SkillHarness, a framework enabling computer-use agents to safely learn and reuse skills in dynamic environments by constraining skill learning against adversarial attacks and environmental disruptions. The system reduces unsafe skill rates by 57.1% compared to existing approaches, addressing a critical vulnerability in AI agents deployed in interactive settings.

Computer-use agents represent an emerging category of AI systems designed to interact with digital environments autonomously, performing tasks across applications and web interfaces. As these agents become more prevalent in production environments, the ability to learn and refine skills from experience becomes essential for practical deployment. However, current skill-learning approaches assume controlled, static environments where agents can safely extract reusable patterns from successful trajectories—an assumption that breaks down in real-world deployments where pop-ups, prompt injections, and other disruptions create safety risks.

SkillHarness addresses this fundamental gap by treating skill learning as a safety-constrained optimization process rather than a simple abstraction exercise. The framework introduces skill boundaries that filter trajectories through multi-source supervision signals, distinguishing between genuinely safe skills and those that merely appear effective in limited contexts. Critically, the system implements selective skill reuse, allowing agents to decompose tasks contextually and activate only appropriate skill subsets for given situations, reducing the attack surface for adversarial manipulation.

For the AI industry, this research validates that safety mechanisms must be built into the skill-learning pipeline itself, not retrofitted afterward. The 57.1% reduction in unsafe skill rates demonstrates measurable progress on a concrete problem affecting real deployments. Organizations developing autonomous agents for customer service, data processing, or system administration will find these techniques directly applicable. The work signals that agent reliability depends on treating dynamic environments as the norm rather than exception, pushing the field toward production-grade safety standards essential for enterprise adoption and regulatory compliance.

• →SkillHarness reduces unsafe skill rates by 57.1% through safety-constrained skill learning in dynamic environments
• →Multi-source supervision signals identify safe skills from trajectories while filtering adversarial interactions and environmental noise
• →Selective skill reuse architecture enables context-aware task decomposition and controlled skill activation
• →Framework addresses critical gap between theoretical skill learning and real-world deployment challenges
• →Safety-first approach to agent skill learning emerging as industry necessity for enterprise applications