🧠 AI🟢 Bullish

Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing

arXiv – CS AI|Justin W. Lin, Eliot Krzysztof Jones, Donovan Julian Jasper, Ethan Jun-shen Ho, Anna Wu, Arnold Tianyi Yang, Neil Perry, Andy Zou, Matt Fredrikson, J. Zico Kolter, Percy Liang, Dan Boneh, Daniel E. Ho|March 4, 2026 at 05:00 AM|1 views

🤖AI Summary

Researchers conducted the first comprehensive evaluation comparing AI agents to human cybersecurity professionals in live penetration testing on a university network with 8,000 hosts. The new ARTEMIS AI agent framework placed second overall, discovering 9 vulnerabilities with 82% accuracy and outperforming 9 of 10 human participants while costing significantly less at $18/hour versus $60/hour for human testers.

Key Takeaways

→ARTEMIS AI agent outperformed 9 out of 10 human cybersecurity professionals in real-world penetration testing.
→AI agents demonstrated cost advantages at $18/hour compared to $60/hour for professional penetration testers.
→ARTEMIS achieved an 82% valid submission rate while discovering 9 valid vulnerabilities in the enterprise environment.
→AI agents excelled at systematic enumeration and parallel exploitation but struggled with GUI-based tasks and had higher false-positive rates.
→Existing AI scaffolds like Codex and CyAgent underperformed compared to most human participants, highlighting the advancement of ARTEMIS.

#ai-agents #cybersecurity #penetration-testing #artemis #enterprise-security #automation #vulnerability-assessment #ai-vs-humans #cost-efficiency #research

Read Original →via arXiv – CS AI

Act on this with AI

Stay ahead of the market.

Connect your wallet to an AI agent. It reads balances, proposes swaps and bridges across 15 chains — you keep full control of your keys.

Connect Wallet to AI →How it works

AI1h ago

Warren Buffett complained for decades that boosting profits by excluding exec stock comp was ‘cynical’—Nvidia just surprised Wall Street and agreed

Nvidia surprised Wall Street by agreeing to include executive stock compensation in its profit calculations, addressing a decades-old complaint by Warren Buffett about excluding such costs. This accounting change will likely boost Nvidia's credibility with investors while potentially pressuring competitors to follow suit.

AI4h ago

A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilities

Researchers introduced NeuroCognition, a new benchmark for evaluating LLMs based on neuropsychological tests, revealing that while models show unified capability across tasks, they struggle with foundational cognitive abilities. The study found LLMs perform well on text but degrade with images and complexity, suggesting current models lack core adaptive cognition compared to human intelligence.

AI4h ago

Estimating Visual Attribute Effects in Advertising from Observational Data: A Deepfake-Informed Double Machine Learning Approach

Researchers developed DICE-DML, a new framework that uses deepfake technology and machine learning to measure causal effects of visual attributes in digital advertising. The method addresses bias issues in standard approaches when analyzing how image elements like skin tone affect consumer engagement on social media platforms.