A Gauge Theory of Superposition: Toward a Sheaf-Theoretic Atlas of Neural Representations

Researchers propose a new gauge-theoretic framework for understanding superposition in large language models, replacing traditional single-dictionary approaches with local semantic charts. The method introduces three measurable obstructions to interpretability and demonstrates results on Llama 3.2 3B model with various datasets.

• →Novel gauge theory framework replaces single-global-dictionary premise with sheaf-theoretic atlas for LLM interpretation.
• →Three key obstructions to global interpretability identified: local jamming, proxy shearing, and nontrivial holonomy.
• →Framework tested on Llama 3.2 3B Instruct model using WikiText-103 and other datasets with non-vacuous certified bounds.
• →Method provides computable gauge-invariant holonomy measures and unavoidable failure bounds for model interpretability.
• →Bootstrap experiments demonstrate stable estimation of shearing and holonomy measures with improved concentration.