A Unified Geometric Space for Topological Alignment Between Transformer-Based Models and Human Brain Networks

Researchers have developed a novel framework for comparing Transformer-based AI models by mapping their internal attention topology onto human brain networks, analyzing 151 models across vision, language, and multimodal domains. The study reveals an arc-shaped distribution of topological alignment with human cognition, where models trained for semantic abstraction align with higher-order brain networks, while detail-focused models align with low-level networks, though alignment scores show weak correlation with standard performance metrics.

This research introduces a brain-referenced topological mapping methodology that transcends traditional AI-human alignment studies by focusing on organizational properties rather than task-specific performance. The work examines how Transformer architectures inherently structure their attention mechanisms in ways that parallel human intrinsic connectivity networks, providing quantitative validation of architectural design principles across different modalities without requiring task-specific inputs.

The findings challenge conventional assumptions about AI model development. The weak correlation between topological alignment and ImageNet-1K accuracy (r=0.266, p=0.156) suggests that brain-like organization doesn't necessarily predict downstream performance on benchmark tasks. The counterintuitive results—such as DINOv2's reduced alignment compared to predecessors and the scaling inversion in distilled DeiT models—indicate that optimization for specific objectives may diverge from biologically-inspired organizational patterns. These discoveries suggest that human brain alignment and task performance represent distinct optimization pathways.

For the AI research community, this framework enables modality-agnostic comparison of model architectures at a fundamental level beyond benchmarks. Developers can use topological alignment scores as a proxy measure for understanding architectural organization, potentially informing future design choices. The non-significant effect of fine-tuning and instruction tuning on alignment indicates that alignment properties emerge during initial training rather than through adaptation.

Future work should explore whether topological alignment predicts performance on tasks requiring abstract reasoning or whether it correlates with other desirable properties like interpretability or robustness. Understanding whether brain-aligned models generalize better across domains remains an open question worth investigating.

• →Transformer models exhibit arc-shaped topological alignment distribution with human brain networks, varying by training objective and model scale.
• →Semantic abstraction-optimized models align more closely with higher-order cognitive networks, while detail-focused models align with low-level networks.
• →Topological alignment shows weak correlation (r=0.266) with ImageNet-1K accuracy, suggesting brain-like organization and task performance represent distinct optimization paths.
• →Fine-tuning and instruction tuning have minimal impact on topological alignment, indicating alignment properties are established during initial training.
• →The framework enables modality-agnostic comparison across vision, language, and multimodal Transformers without task-specific constraints.