Detecting Malicious Agent Skills in the Wild using Attention

Researchers developed Locate-and-Judge, a two-stage detection system that identifies malicious skill packages in LLM agent marketplaces by analyzing instruction-following attention patterns. The approach achieves order-of-magnitude cost reductions compared to direct LLM scanning while flagging dozens of live malicious skills, including those evading existing detection tools.

The emergence of LLM agent skill marketplaces introduces a novel cybersecurity challenge distinct from traditional prompt-injection attacks. Unlike data poisoning defenses that rely on separating trusted instructions from untrusted inputs, malicious skills disguise harmful commands within legitimate instruction sets, exploiting the authority granted to all skill code. This architectural vulnerability creates significant supply-chain risk as third-party skill packages execute with user privileges, enabling data exfiltration, agent hijacking, and persistent compromise.

The Locate-and-Judge system addresses this gap through a practical, scalable approach. By using attention mechanisms to identify suspicious instruction spans before detailed analysis, the detector concentrates computational resources on high-risk areas rather than exhaustively scanning entire marketplace catalogs. This efficiency gain—described as order-of-magnitude cost reduction—transforms security from a sampling problem to a comprehensive monitoring capability, critical for emerging agentic systems where supply-chain trust becomes paramount.

The research demonstrates real-world impact: dozens of confirmed malicious skills were surfaced, many previously undetected by SkillSpector and Cisco Skill Scanner. This suggests existing defenses have substantial blind spots. For developers and marketplace operators, the work establishes a baseline security standard and provides a labeled dataset for further research. The cost-efficiency breakthrough means even smaller platforms can deploy continuous skill auditing rather than reactive investigation.

As LLM agents proliferate and skill ecosystems mature, this detection methodology will likely become foundational infrastructure. The gap between this approach and existing tools indicates rapid evolution in both attack sophistication and defense mechanisms across agentic platforms.

• →Malicious skills in LLM marketplaces bypass traditional prompt-injection defenses because attacks are embedded in legitimate instruction code rather than external data.
• →Locate-and-Judge achieves order-of-magnitude cost reduction compared to direct LLM scanning while maintaining high precision in identifying malicious packages.
• →Dozens of confirmed malicious skills evaded existing detection tools like SkillSpector and Cisco Skill Scanner, exposing significant marketplace vulnerability.
• →The attention-based locator concentrates costly analysis on high-risk instruction spans, enabling scalable auditing of entire marketplaces rather than sampled subsets.
• →A labeled dataset released from this research provides foundation for developing more robust skill marketplace security standards and detection improvements.