Disentangling Recall and Reasoning in Transformer Models through Layer-wise Attention and Activation Analysis

Researchers used mechanistic interpretability techniques to demonstrate that transformer language models have distinct but interacting neural circuits for recall (retrieving memorized facts) and reasoning (multi-step inference). Through controlled experiments on Qwen and LLaMA models, they showed that disabling specific circuits can selectively impair one ability while leaving the other intact.

• →Transformer models have separable neural circuits for recall and reasoning tasks that can be identified and manipulated independently.
• →Disabling recall circuits reduced fact-retrieval accuracy by up to 15% while preserving reasoning capabilities.
• →The research provides first causal evidence of functional specialization in transformer architecture through layer-wise analysis.
• →Findings could inform safer AI deployment by enabling targeted interventions that preserve desired capabilities.
• →Study advances mechanistic interpretability by linking circuit-level structure to specific cognitive functions in language models.