Regimes: An Auditable, Held-Out-Gated Improvement Loop Demonstrated on LongMemEval with ActiveGraph

Researchers introduce Regimes, an auditable autonomous improvement loop built on the ActiveGraph event-sourced runtime that enables transparent, reproducible AI agent optimization. The system diagnoses failures, proposes repairs, and validates them through multiple gates before promotion, demonstrating 5-10% held-out accuracy improvements on long-context reading comprehension tasks.

This research addresses a fundamental trust problem in autonomous AI improvement systems: the black-box nature of how agents are optimized. Traditional improvement loops operate outside the agent's core logic, creating audit gaps where failures vanish and decisions become invisible. Regimes inverts this architecture by making improvement itself a first-class workflow within an event-sourced runtime where every action leaves an immutable trail.

The technical contribution centers on ActiveGraph, which treats agent state as a deterministic projection of an append-only event log. This design enables exact replay of any run, auditable gate decisions, and failure tracking that stays within the agent's native history rather than scattered across external systems. The approach mirrors database transaction patterns applied to AI workflows, bringing decades of proven audit methodology to autonomous systems.

The LongMemEval-S experiments reveal something counterintuitive: when long-context tasks fail, the bottleneck often isn't information retrieval but reasoning over assembled evidence. Regimes discovered reader-prompt repairs that improved accuracy by 5-10 percentage points on held-out test sets, suggesting that discovery-driven optimization can surface non-obvious failure patterns. The framework's target-agnostic design means the same control flow applies across different tasks through unified interfaces.

For AI infrastructure development, this work signals growing maturity in treating improvement loops as engineering problems rather than research afterthoughts. The emphasis on auditability and gating creates pathways for regulated AI deployment where stakeholders need transparent optimization records. The main open question—whether routing failures to specific pipeline locations adds meaningful value—suggests the framework itself may be more important than the specific improvements demonstrated.

• →Event-sourced runtimes transform AI improvement from external scaffolding into auditable first-class workflows with full replay capability.
• →ActiveGraph's append-only event log design enables transparent decision tracking, gate auditability, and failure diagnosis impossible in traditional agent architectures.
• →Long-context task failures stem primarily from reasoning/reconciliation rather than retrieval, indicating discovery-driven optimization surfaces non-obvious bottlenecks.
• →Regimes achieved 5-10% held-out accuracy improvements on reading comprehension through prompt-repair discovery and multi-stage validation gates.
• →The framework's target-agnostic design and transparent audit trail position it as infrastructure for regulated AI deployment requiring optimization transparency.