Researchers present Neural Information Causality (Neural-IC), a theoretical framework that formalizes how neural network representations function as communication channels under query-separated computation. The work establishes operational bounds on information leakage through bottlenecks and demonstrates that quantum advantages in specific architectures depend on fair query-conditioned access rather than total information capacity.
Neural Information Causality addresses a fundamental problem in representation learning: when data must be encoded before a query is known, the intermediate representation becomes a communication channel with measurable constraints. This distinction between feature maps and operational messages is more than semantic—it enables precise diagnosis of where neural networks leak information.
The framework's key contribution separates two independent principles. First, query-separated architectures mathematically induce random-access communication experiments with provable bounds. Second, physical constraints on the interface—whether bit-width limitations, precision bounds, or noisy channels—directly limit information flow. This separation transforms capacity from a post hoc observation into a predictive diagnostic tool.
The research validates these bounds through controlled experiments, including a classical one-bit benchmark and analysis of CHSH-type correlation layers used in quantum-inspired neural architectures. The finding that Tsirelson thresholds emerge from stability requirements at one-bit bottlenecks suggests quantum advantages depend on structured query access rather than raw capacity. The extension to asymmetric biases and correlated data broadens applicability across diverse architectures.
For machine learning practitioners, Neural-IC provides concrete methods to identify three failure modes: query leakage (unintended information exposure), precision leakage (from finite registers), and memory inefficiency. The framework's grounding in communication theory makes it applicable to bottleneck design in transformers, compression networks, and privacy-preserving systems. The controlled ablations demonstrating that apparent violations correspond to broken assumptions strengthen confidence in the theoretical predictions.
- →Neural-IC formalizes how intermediate representations function as communication channels with quantifiable information bounds
- →Physical constraints on network interfaces directly limit information flow independent of architectural complexity
- →Quantum advantages in correlation layers stem from fair query-conditioned access, not total information beyond bottlenecks
- →The framework provides operational diagnostics for query leakage, precision loss, and memory inefficiency in neural networks
- →Controlled experiments validate theoretical predictions and identify failure modes from broken query separation