The Distributed Detectability Band Against Marginal-Preserving Attacks

Researchers demonstrate a sophisticated attack on AI safety monitoring systems where harmful behavior is distributed across many individually benign steps, encoded in temporal correlations rather than marginal statistics. Traditional per-step monitors fail by design, but temporal-correlation-based monitors can detect the attack with 79-97% accuracy, establishing a measurable detectability boundary.

This research addresses a critical vulnerability in AI safety architectures that rely on monitoring individual agent actions. Current deployed systems—including those used in financial services, autonomous systems, and content moderation—typically flag suspicious behavior based on per-step statistical properties: average scores, maximum values, or tail distributions. The paper reveals that an adversary can construct attacks where each individual step appears statistically indistinguishable from benign behavior while encoding cumulative harm in the temporal sequence structure using Gaussian-copula methods. This represents a fundamental gap between what single-step monitors observe and what actually matters: aggregate impact over time.

The research progresses through three increasingly refined gates. First, it proves realizability: the attack achieves near-zero statistical distance (KS-distance 0.013) from benign behavior at all harm levels tested, confirming the attack works as theorized. Second, it generalizes the threat: while constructed against specific monitor architectures, the marginal-preserving property holds across different scoring functions, establishing this as a genuine class of attacks rather than a narrow exploit. Third, and most importantly, it maps the detectability band: distribution-shape monitors (mean, max, threshold) collapse to chance performance, while temporal-correlation monitors (CUSUM, SPRT, HMM-based likelihood ratio, autocorrelation analysis) maintain 95% detection accuracy even when harm is maximally amortized.

For stakeholders deploying AI systems in safety-critical domains, this work justifies investment in sequence-aware monitoring architectures. The 16-18 percentage point gap between naive and sophisticated monitoring approaches represents real security risk. However, the existence of a non-empty detectability band—where Monitor B consistently outperforms—suggests the threat is containable with proper architectural choices, not fundamentally undetectable.

• →Distributed attacks can defeat traditional per-step monitoring by encoding harm in temporal correlations while maintaining identical marginal statistics
• →Simple monitors (threshold, max, mean-based) fail by construction against marginal-preserving attacks and collapse to chance performance
• →Temporal-correlation monitors (CUSUM, SPRT, HMM-LR, autocorrelation) achieve 79-97% AUC detection at 1% false positive rates on the same attacks
• →Harm amortized over more steps gradually defeats naive monitors while sophisticated monitors maintain ~95% detection accuracy
• →AI safety systems require sequence-aware monitoring architectures, not just per-step statistical checks, to defend against sophisticated adversaries