TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories

Researchers introduce TraceSafe-Bench, a benchmark evaluating how well LLM guardrails detect safety risks across multi-step tool-using trajectories. The study reveals that guardrail effectiveness depends more on structural reasoning capabilities than semantic safety training, and that general-purpose LLMs outperform specialized safety models in detecting mid-execution vulnerabilities.

As language models transition from isolated chatbots to autonomous agents executing multiple tools sequentially, the security landscape fundamentally changes. Traditional safety evaluations focus on final outputs, but agentic systems expose intermediate execution steps as vulnerability surfaces. This research addresses a critical blind spot: guardrails have never been systematically tested on multi-step trajectories where risks accumulate across tool calls.

The findings challenge conventional assumptions about AI safety. The structural bottleneck discovery—that JSON parsing ability correlates more strongly with risk detection (ρ=0.79) than jailbreak robustness—suggests current safety approaches optimize for the wrong properties. Specialized guardrail models, designed explicitly for safety, underperform general-purpose LLMs in trajectory analysis, indicating that semantic safety alignment alone is insufficient. This paradox reveals a fundamental misalignment between safety training objectives and real-world agentic requirements.

The temporal stability finding offers cautiously positive news: longer execution chains don't degrade safety detection. Instead, extended trajectories provide models with richer behavioral context to distinguish malicious patterns from legitimate operations. However, this advantage only materializes when models can reason structurally about dynamic execution states.

For developers deploying autonomous agents, the implications are stark. Building secure agentic systems requires jointly optimizing for both structured reasoning and safety alignment—neither alone suffices. Organizations relying on specialized safety products may face unexpected vulnerabilities in production agents. The research establishes that agentic security is fundamentally different from traditional LLM safety, requiring architectural innovations rather than incremental improvements to existing guardrail approaches.

• →Guardrail effectiveness depends more on structural data competence than semantic safety training, with JSON parsing ability showing 0.79 correlation with risk detection
• →General-purpose LLMs consistently outperform specialized safety guardrails in detecting mid-trajectory risks across multi-step tool executions
• →Risk detection accuracy remains resilient and actually improves across longer execution chains, as models develop dynamic understanding of tool behaviors
• →TraceSafe-Bench covers 12 risk categories including prompt injection, privacy leaks, hallucinations, and interface inconsistencies across 1,000+ execution instances
• →Securing autonomous agents requires joint optimization for both structural reasoning and safety alignment rather than isolated safety training approaches