Integrated and Cross-Architecture Interpretation of LLM Reasoning

Researchers present the Integrated cross-Architecture Reasoning (IAR) framework, a novel methodology for interpreting how large language models perform reasoning tasks by combining multiple analytical probes—bandwidth-calibrated Mutual Information Peak, Deep-Thinking Ratio analysis, and Jaccard stability metrics—across model layers and architectures. Testing on Qwen and Llama models across mathematics, code, logic, and common sense domains demonstrates that this multi-metric approach provides more reliable insights into LLM reasoning patterns than single-probe methods.

The opacity of LLM reasoning processes presents a fundamental challenge for AI developers and researchers seeking to build more reliable, interpretable systems. Existing single-metric approaches like Mutual Information Peak or Deep-Thinking Ratio provide incomplete pictures of how models generate outputs, risking mischaracterization of underlying reasoning mechanisms. The IAR framework addresses this limitation by synthesizing multiple complementary analytical techniques into a unified interpretability approach.

LLM interpretability has become increasingly critical as these models assume larger roles in high-stakes decision-making. Understanding whether models genuinely reason or merely pattern-match affects confidence in their deployment across domains from mathematics to code generation. Prior work isolated reasoning-crucial tokens, but lacked systematic methods to validate findings across layers and architectures, leaving gaps in understanding how reasoning patterns evolve during computation.

The IAR framework's cross-architecture testing on both Qwen variants and Llama models across diverse problem domains signals meaningful progress toward generalizable interpretability tools. The overlap analysis between reasoning tokens and computation-intensive tokens reveals whether reasoning requires substantial computational resources, offering insights for model optimization and efficiency improvements. The Jaccard stability metric provides validation that identified tokens genuinely correlate with reasoning quality rather than spurious correlations.

For the AI development community, this research establishes methodological foundations for more rigorous interpretability studies. As enterprises evaluate LLM deployment and regulators demand transparency, frameworks like IAR become essential infrastructure. Future work likely extends these techniques to larger models and explores whether insights improve model training or debugging processes.

• →IAR framework combines three analytical techniques to overcome limitations of single-metric LLM reasoning interpretation.
• →Cross-architecture validation on Qwen-7B, Qwen-14B, and Llama-8B demonstrates generalizability across model families.
• →Overlap analysis between reasoning tokens and computation-intensive tokens reveals computational costs of reasoning processes.
• →Jaccard stability metric validates that identified tokens correlate with genuine reasoning quality across multiple domains.
• →Multi-domain testing spanning mathematics, code, logic, and common sense strengthens framework applicability beyond narrow use cases.