6 articles tagged with #policy-compliance. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
LedgerAgent is a new inference-time method that improves how AI agents handle customer-service tasks by maintaining explicit task states in a separate ledger rather than reconstructing context from prompts. The approach reduces policy violations and improves decision consistency across multiple trials by validating state-dependent constraints before executing tool calls.
Researchers introduced RuleSafe-VL, a new benchmark for evaluating how well vision-language AI models apply explicit content moderation rules. The benchmark reveals significant gaps in rule-reasoning capabilities, with even top models achieving only 64.8% accuracy on rule-interaction recovery, indicating current safety systems may reach correct moderation decisions through superficial pattern-matching rather than genuine policy understanding.
Researchers identified a critical failure mode in LLM-based agents called policy-invisible violations, where agents execute actions that appear compliant but breach organizational policies due to missing contextual information. They introduced PhantomPolicy, a benchmark with 600 test cases, and Sentinel, an enforcement framework using counterfactual graph simulation that achieved 93% accuracy in detecting violations compared to 68.8% for baseline approaches.
Researchers introduce NormBench, a benchmark with 2,290 legal provisions across multiple languages, and Span-Grounded Deontic Trees (SG-DT), a structured representation method designed to address Silent Scope Omission—where AI systems appear compliant but fail to apply nested exceptions correctly. Testing reveals that frontier LLMs struggle with recursive defeater chains and struggle to assemble correct logical control flow despite retrieving relevant source material.
Researchers present SKG-VLA, an AI system that uses Scene Knowledge Graphs to improve decision-making in large-scale complaint handling by integrating multimodal evidence (text, images, metadata) with structured reasoning about entities, policies, and temporal events. The approach demonstrates improved accuracy and robustness across policy-grounded reasoning and long-tail scenarios.
Researchers introduce EmbodiedGovBench, a new evaluation framework for embodied AI systems that measures governance capabilities like controllability, policy compliance, and auditability rather than just task completion. The benchmark addresses a critical gap in AI safety by establishing standards for whether robot systems remain safe, recoverable, and responsive to human oversight under realistic failures.
