The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws

Researchers demonstrate that sparse autoencoders (SAEs) used to interpret AI model activations face fundamental geometric constraints rather than just resource limitations. By analyzing 844 SAE checkpoints across Gemma 2 models, they show that manifold curvature and intrinsic dimensionality at each layer predict reconstruction performance, establishing a transferable geometric law that explains why SAE effectiveness varies across layers.

This research addresses a fundamental challenge in AI interpretability: understanding why sparse autoencoders perform inconsistently across different layers of neural networks. Rather than attributing this to insufficient model capacity, the authors propose that the geometric structure of activation spaces themselves creates an irreducible reconstruction floor. The study represents significant progress in mechanistic interpretability by connecting abstract mathematical properties to practical scaling behavior.

The work builds on the linear representation hypothesis, which assumes that neural network activations can be reconstructed as sparse linear combinations of interpretable features. However, this assumption breaks down when the underlying activation manifold is curved or has varying complexity across layers. The researchers conducted an extensive empirical study using Gemma 2 models at multiple scales, fitting scaling laws at individual layers and then analyzing how geometric properties predict performance variation.

The discovery that manifold geometry predicts SAE behavior across different models has profound implications for AI interpretability research. It suggests that geometric properties are fundamental constraints rather than model-specific artifacts, enabling researchers to anticipate interpretability challenges before encountering them. This understanding could inform architecture design decisions and help practitioners allocate resources more effectively when attempting to interpret large language models.

The transferability of geometric insights between different model scales indicates a deep structural principle in neural network organization. Future work may focus on whether these geometric constraints apply to other interpretability methods or whether they suggest alternative approaches that better accommodate curved manifold structures. This research advances the theoretical foundation of mechanistic interpretability from empirical observation toward principled geometric understanding.

• →SAE reconstruction performance is constrained by activation manifold geometry, not just model width or sparsity parameters
• →Higher curvature and intrinsic dimensionality in activation spaces create irreducible reconstruction floors that no sparse linear model can overcome
• →Geometric scaling laws transfer across different model scales, suggesting universal principles governing neural network interpretability
• →Per-layer width exponents can be predicted from manifold geometric summaries, enabling principled scaling law design
• →Current SAE limitations reflect fundamental geometric properties rather than resource constraints