Moving Beyond Diffusion: Hierarchy-to-Hierarchy Autoregression for fMRI-to-Image Reconstruction

Researchers introduce MindHier, a new framework for reconstructing visual images from brain fMRI signals using hierarchical autoregressive modeling instead of diffusion methods. The approach achieves 4.67x faster inference while improving semantic accuracy by aligning neural hierarchies with image generation stages, mimicking human visual perception.

MindHier represents a meaningful advancement in brain-computer interface research by addressing fundamental inefficiencies in existing diffusion-based fMRI reconstruction methods. Previous approaches collapsed hierarchical brain information into static guidance, creating a mismatch between how the brain processes visual information and how images are generated. This new framework recognizes that visual perception operates hierarchically—from broad semantic understanding to fine details—and structures the reconstruction process accordingly.

The technical innovation centers on three key components: extracting multi-level neural embeddings from fMRI data, aligning these embeddings with CLIP features layer-by-layer, and strategically injecting guidance at matching scales during image generation. This hierarchical approach yields substantial practical improvements beyond raw performance metrics. The 4.67x inference speedup has real implications for real-time brain-computer interfaces, while the more deterministic results reduce variance in reconstruction quality.

For the neuroscience and AI community, this work bridges a critical gap between cognitive neuroscience and modern deep learning architectures. It demonstrates that respecting biological constraints and hierarchical information flow yields better technical outcomes. The NSD dataset validation provides credibility within the research community. However, this remains fundamental research without immediate commercial applications. The implications matter most for future medical applications like assistive technologies for paralyzed patients or brain-computer communication systems, which remain years from market deployment. The efficiency gains and cognitive alignment suggest this approach could accelerate progress toward practical neural interfaces.

• →MindHier achieves 4.67x faster inference and improved semantic accuracy compared to diffusion-based fMRI reconstruction methods
• →Hierarchical alignment of brain activity with image generation stages mirrors human visual perception architecture
• →Scale-aware guidance injection at matching network layers creates more cognitively aligned reconstruction processes
• →Results are more deterministic than diffusion baselines, reducing reconstruction variance and improving reliability
• →Advancement addresses fundamental inefficiencies in static guidance approaches to brain-to-image synthesis