#information-bottleneck News & Analysis

4 articles tagged with #information-bottleneck. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

4 articles

AIBullisharXiv – CS AI · Mar 267/10

🧠

Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoning

Researchers introduce Bottlenecked Transformers, a new architecture that improves AI reasoning by up to 6.6 percentage points through periodic memory consolidation inspired by brain processes. The system uses a Cache Processor to rewrite key-value cache entries at reasoning step boundaries, achieving better performance on math reasoning benchmarks compared to standard Transformers.

AINeutralarXiv – CS AI · Apr 206/10

🧠

VIB-Probe: Detecting and Mitigating Hallucinations in Vision-Language Models via Variational Information Bottleneck

Researchers propose VIB-Probe, a novel framework using Variational Information Bottleneck theory to detect and mitigate hallucinations in Vision-Language Models by analyzing internal attention mechanisms. The method identifies specific attention heads responsible for truthful generation and introduces an inference-time intervention strategy that outperforms existing detection baselines.

AINeutralarXiv – CS AI · Mar 126/10

🧠

Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneck

Researchers introduce DIBJudge, a new framework to address systematic bias in large language models that favor machine-translated text over human-authored content in multilingual evaluations. The solution uses variational information compression to isolate bias factors and improve LLM judgment accuracy across languages.

AINeutralLil'Log (Lilian Weng) · Sep 286/10

🧠

Anatomize Deep Learning with Information Theory

Professor Naftali Tishby applied information theory to analyze deep neural network training, proposing the Information Bottleneck method as a new learning bound for DNNs. His research identified two distinct phases in DNN training: first representing input data to minimize generalization error, then compressing representations by forgetting irrelevant details.