AINeutralarXiv – CS AI · 2d ago7/10
🧠Researchers introduce OpenClawBench, a large-scale dataset of 31,264 annotated agent execution trajectories that reveals a significant gap between task success and process reliability. The study finds that 9.3% of oracle-passing executions contain process-side anomalies like unresolved ambiguities and unsafe operations, demonstrating that success metrics alone mask critical failure modes in AI agent systems.
AIBullisharXiv – CS AI · 2d ago7/10
🧠Researchers introduce GRASP, a method for improving large language model agents through controlled skill library updates that prevent performance regression. Tested across five base models on clinical benchmarks, GRASP achieves dramatic improvements (40.6% to 88.8% on MedAgentBench) while maintaining stability, outperforming existing self-improvement approaches by significant margins.
🧠 GPT-4🧠 GPT-5🧠 Gemini
AIBearisharXiv – CS AI · 2d ago7/10
🧠Researchers discovered that reflexive AI agents systematically store confident but false interpretations of tasks in their memory, a phenomenon called memory confabulation, causing them to repeat incorrect behaviors even when environments reset. The study introduces a metric to detect this failure mode and proposes programmatic solutions that significantly improve agent performance and reduce reliance on false reflective content.
AIBullisharXiv – CS AI · 3d ago7/10
🧠ReflexGrad introduces a dual-process architecture enabling LLM agents to recover from failures within a single episode without requiring demonstrations. The system combines fast continuous refinement with slow causal diagnosis, achieving significant performance improvements on benchmark tasks with smaller models matching larger model performance through architectural innovation rather than scale.
🧠 GPT-5
AINeutralarXiv – CS AI · Apr 157/10
🧠Researchers introduce HORIZON, a diagnostic benchmark for identifying and analyzing why large language model agents fail at long-horizon tasks requiring extended action sequences. By evaluating state-of-the-art models across multiple domains and proposing an LLM-as-a-Judge attribution pipeline, the study provides systematic methodology for understanding agent limitations and improving reliability.
🧠 GPT-5🧠 Claude
AIBullisharXiv – CS AI · Apr 147/10
🧠Researchers introduce DiaFORGE, a three-stage framework for training LLMs to reliably invoke enterprise APIs by focusing on disambiguation between similar tools and underspecified arguments. Fine-tuned models achieved 27-49 percentage points higher tool-invocation success than GPT-4o and Claude-3.5-Sonnet, with an open corpus of 5,000 production-grade API specifications released for further research.
🧠 GPT-4🧠 Claude
AIBullisharXiv – CS AI · Mar 56/10
🧠Researchers propose a dual-helix governance framework to address AI agent reliability issues in WebGIS development, implementing a 3-track architecture that achieved 51% reduction in code complexity. The framework uses knowledge graphs and self-learning cycles to overcome LLM limitations like context constraints and instruction failures.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers introduce OR-Space, a comprehensive benchmark for evaluating large language model agents in industrial operations research workflows. Unlike existing benchmarks that focus on single-stage problem translation, OR-Space tests agents across persistent multi-artifact workspaces with three task modes—building optimization models, revising them under changing requirements, and explaining solutions—to assess real-world reliability and practical readiness.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers propose FeasiGen, a framework for automatically generating infeasible task benchmarks to evaluate whether AI agents recognize when tasks cannot be completed with available tools. Testing across nine models reveals critical weaknesses, with agents continuing execution on impossible tasks up to 73.9% of the time, though multi-agent architectures show improved performance.
AIBullisharXiv – CS AI · 4d ago6/10
🧠Researchers introduce NoisyAgent, a training framework that improves large language model agent robustness by deliberately exposing them to environmental imperfections during training. By simulating real-world interaction noise—including user ambiguity and tool failures—the approach bridges the gap between idealized benchmark performance and practical deployment reliability.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers introduce VIGIL, an evaluation framework that separately measures whether embodied AI agents correctly complete tasks and properly report success, rather than conflating execution failures with commitment failures. Testing across 20 models reveals significant performance gaps in terminal commitment despite similar task execution, highlighting a critical blind spot in current AI agent benchmarking.
AINeutralarXiv – CS AI · May 96/10
🧠Researchers analyzed internal mechanisms of LLM-based agent memory systems across the Qwen model family, discovering that routing circuits activate before content extraction circuits—a critical gap in small models. They developed an unsupervised diagnostic tool achieving 76.2% accuracy in identifying where silent memory failures occur, providing practical insights for improving agent reliability.
AINeutralarXiv – CS AI · Apr 146/10
🧠ClawVM is a virtual memory management system designed for stateful LLM agents that addresses critical failures in current context window management. The system implements typed pages, multi-resolution representations, and validated writeback protocols to ensure deterministic state residency and durability, adding minimal computational overhead.
AIBullisharXiv – CS AI · Mar 27/1011
🧠Researchers propose a new framework for foundation world models that enables autonomous agents to learn, verify, and adapt reliably in dynamic environments. The approach combines reinforcement learning with formal verification and adaptive abstraction to create agents that can synthesize verifiable programs and maintain correctness while adapting to novel conditions.
