#attention-circuits News & Analysis

Spectral Probe-Circuits: A Three-Step Recipe for Identifying Attention-Head Circuits in Pretrained Transformers

Researchers present a three-step methodology for identifying and validating attention-head circuits in transformer models using spectral analysis, pattern filtering, and causal ablation. The technique successfully isolates core computational circuits across multiple model sizes and architectures without requiring labeled data or gradient attribution.