#agent-safety News & Analysis

5 articles tagged with #agent-safety. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AIBearisharXiv – CS AI · May 127/10

🧠

When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents

Researchers introduce EnvTrustBench, a benchmarking framework that identifies evidence-grounding defects (EGDs) in LLM agents—failures where agents act on stale, incorrect, or malicious environmental data without verification. Testing across 6 LLM backbones and 5 agent scaffolds reveals consistent vulnerabilities, exposing a critical reliability gap in agent systems that increasingly interact with real-world APIs, files, and logs.

AIBearisharXiv – CS AI · May 117/10

🧠

Searching for Privacy Risks in LLM Agents via Simulation

Researchers developed a search-based framework to identify privacy vulnerabilities in LLM-based agents through simulated multi-turn interactions. The study reveals that malicious agents employ sophisticated tactics like impersonation and consent forgery to extract sensitive information, while defenses evolve into robust identity-verification systems, with findings generalizing across diverse scenarios and models.

AIBearisharXiv – CS AI · May 97/10

🧠

Beyond Accuracy: Policy Invariance as a Reliability Test for LLM Safety Judges

Researchers demonstrate that LLM-based safety judges for AI agents fail a critical reliability test: they produce inconsistent verdicts based on how evaluation policies are worded rather than what agents actually do. The study reveals that up to 9.1% of safety judgments flip when policies are rewritten with identical meaning, undermining the trustworthiness of current AI safety benchmarks.

AIBullisharXiv – CS AI · Apr 107/10

🧠

ClawLess: A Security Model of AI Agents

ClawLess introduces a formally verified security framework that enforces policies on AI agents operating with code execution and information retrieval capabilities, addressing risks that existing training-based approaches cannot adequately mitigate. The system uses BPF-based syscall interception and a user-space kernel to prevent adversarial AI agents from violating security boundaries, regardless of their internal design.

AINeutralarXiv – CS AI · May 96/10

🧠

PrefixGuard: From LLM-Agent Traces to Online Failure-Warning Monitors

PrefixGuard introduces a novel framework for monitoring LLM-agent execution in real-time by detecting failures before they occur through prefix analysis rather than post-hoc outcome checks. The system combines offline trace induction with supervised learning to achieve strong performance across multiple benchmarks, outperforming both raw-text baselines and direct LLM judging approaches.