Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoning

Researchers introduce Bottlenecked Transformers, a new architecture that improves AI reasoning by up to 6.6 percentage points through periodic memory consolidation inspired by brain processes. The system uses a Cache Processor to rewrite key-value cache entries at reasoning step boundaries, achieving better performance on math reasoning benchmarks compared to standard Transformers.

• →New Bottlenecked Transformer architecture improves reasoning performance by up to 6.6 percentage points on math benchmarks.
• →The approach uses brain-inspired memory consolidation processes to rewrite KV cache entries during reasoning steps.
• →Information Bottleneck theory provides theoretical justification for why KV cache rewrites improve model generalization.
• →The Cache Processor performs periodic, non-causal rewrites at reasoning step boundaries to consolidate memory traces.
• →This represents a novel approach to Auxiliary Latent-Space Computation that outperforms pause-token augmented baselines.