#information-bottleneck News & Analysis

7 articles tagged with #information-bottleneck. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

7 articles

AIBullisharXiv – CS AI · Mar 267/10

🧠

Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoning

Researchers introduce Bottlenecked Transformers, a new architecture that improves AI reasoning by up to 6.6 percentage points through periodic memory consolidation inspired by brain processes. The system uses a Cache Processor to rewrite key-value cache entries at reasoning step boundaries, achieving better performance on math reasoning benchmarks compared to standard Transformers.

AIBullisharXiv – CS AI · Jun 86/10

🧠

Dual Latent Memory for Visual Multi-agent System

Researchers propose L²-VMAS, a framework addressing the 'scaling wall' problem in Visual Multi-Agent Systems where adding more agents degrades performance despite higher computational costs. The solution uses dual latent memory and entropy-driven triggering to improve accuracy by 2.7-5.4% while reducing token usage by 21.3-44.8%.

AINeutralarXiv – CS AI · Jun 56/10

🧠

Beyond Soft Masks: Hard-Perturbation Mixup Explainer for Robust GNN Explainability

Researchers propose HPME, a novel framework for explaining Graph Neural Network decisions using hard-perturbation mixup strategies instead of soft masks. The method addresses out-of-distribution issues in GNN explainability by extracting discrete subgraphs and employing structure-level replacement, achieving improved explanation fidelity across synthetic and real-world datasets.

AINeutralarXiv – CS AI · May 16/10

🧠

Why Self-Supervised Encoders Want to Be Normal

Researchers develop a theoretical framework connecting Information Bottleneck principles to encoder-decoder learning through rate-distortion analysis, showing optimal representations form soft clusters on probability manifolds. The work introduces Sketched Isotropic Gaussian Regularization (SIGReg) as a principled regularizer for self-supervised, semi-supervised, and supervised learning without requiring variational bounds.

AINeutralarXiv – CS AI · Apr 206/10

🧠

VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck

Researchers propose VIB-Probe, a novel framework using Variational Information Bottleneck theory to detect and mitigate hallucinations in Vision-Language Models by analyzing internal attention mechanisms. The method identifies specific attention heads responsible for truthful generation and introduces an inference-time intervention strategy that outperforms existing detection baselines.

AINeutralarXiv – CS AI · Mar 126/10

🧠

Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneck

Researchers introduce DIBJudge, a new framework to address systematic bias in large language models that favor machine-translated text over human-authored content in multilingual evaluations. The solution uses variational information compression to isolate bias factors and improve LLM judgment accuracy across languages.

AINeutralLil'Log (Lilian Weng) · Sep 286/10

🧠

Anatomize Deep Learning with Information Theory

Professor Naftali Tishby applied information theory to analyze deep neural network training, proposing the Information Bottleneck method as a new learning bound for DNNs. His research identified two distinct phases in DNN training: first representing input data to minimize generalization error, then compressing representations by forgetting irrelevant details.