Closure-Validated Circuit Discovery in Attention Heads: Co-activation Proposes, Ablation Disposes

Researchers propose a methodology for validating attention-head circuits in large language models by combining co-activation clustering with causal ablation testing. Their findings reveal that while clustering signals identify circuit proposals, true circuit validation requires closure tests that measure functional impact through ablation—a distinction that challenges current interpretability approaches.

This research addresses a fundamental challenge in AI interpretability: distinguishing between correlative signals and genuinely functional circuits within neural networks. The team develops a validation framework that moves beyond reconstruction-based metrics, which have dominated circuit discovery work. By applying causal ablation—the gold standard for functional validation—across multiple model architectures, they establish empirical evidence that cheap co-activation signals alone cannot confirm circuit discovery.

The work emerged from growing recognition that interpretability research often conflates correlation with causation. Previous methods clustered co-activated components but rarely validated whether these clusters performed meaningful functions. This research fills that gap by testing whether ablating discovered communities actually degrades model performance in predictable ways.

The results vary significantly by architecture. Dense models (Pythia 1B and OLMo 1B) show communities that survive closure validation, suggesting genuine functional circuits exist at this scale. However, the Mixture-of-Experts model presents a cautionary finding: route-conditional clustering recovered statistically significant patterns that failed closure tests and bizarrely improved loss when ablated. This divergence indicates that architectural properties fundamentally shape circuit organization and that validation methods must adapt accordingly.

These findings matter for AI safety and interpretability research. As models grow more complex, developing reliable methods for identifying functional components becomes critical for understanding failure modes and ensuring alignment. The work establishes closure as a necessary validation step rather than optional confirmation, raising methodological standards across the field and suggesting that many previously published circuit discoveries may require re-evaluation.

• →Co-activation clustering identifies circuit proposals but requires causal ablation validation to confirm actual functional circuits.
• →Dense language models show statistically validated circuits through closure testing, while mixture-of-experts models display decoupling between activation patterns and function.
• →Current interpretability methods often conflate correlation with causation, necessitating rigorous causal validation frameworks.
• →Architectural differences significantly impact circuit organization, requiring validation approaches adapted to specific model types.
• →The methodology establishes closure testing as a critical standard for circuit discovery research across AI interpretability.