34 articles tagged with #information-theory. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers present a unified theoretical framework for understanding generative diffusion models by connecting information theory, dynamics, and thermodynamics. The study reveals that diffusion generation operates as controlled noise-induced symmetry breaking, where the score function regulates information flow from noise to structured data.
Researchers developed an information-theoretic framework to explain 'Aha moments' in large language models during reasoning tasks. The study reveals that strong reasoning performance stems from uncertainty externalization rather than specific tokens, decomposing LLM reasoning into procedural information and epistemic verbalization.
Researchers introduce InfoPO (Information-Driven Policy Optimization), a new method that improves AI agent interactions by using information-gain rewards to identify valuable conversation turns. The approach addresses credit assignment problems in multi-turn interactions and outperforms existing baselines across diverse tasks including intent clarification and collaborative coding.
Researchers introduce 'bipredictability' as a new metric to monitor reinforcement learning agents in real-world deployments, measuring interaction effectiveness through shared information ratios. The Information Digital Twin (IDT) system detects 89.3% of perturbations versus 44% for traditional reward-based monitoring, with 4.4x faster detection speed.
Researchers introduced Entanglement Learning (EL), an information-theoretic framework that enhances Model Predictive Control (MPC) for autonomous systems like UAVs. The framework uses an Information Digital Twin to monitor information flow and enable real-time adaptive optimization, improving MPC reliability beyond traditional error-based feedback systems.
Professor Naftali Tishby applied information theory to analyze deep neural network training, proposing the Information Bottleneck method as a new learning bound for DNNs. His research identified two distinct phases in DNN training: first representing input data to minimize generalization error, then compressing representations by forgetting irrelevant details.
The article provides an information-theoretic analysis of brainwallets, which store cryptocurrency funds using private keys generated from memorized passwords. While brainwallets theoretically offer strong security for long-term storage, they remain controversial due to practical implementation challenges and potential vulnerabilities.
Researchers introduce EAGLE, a new framework for explaining black-box machine learning models using information-theoretic active learning to select optimal data perturbations. The method produces feature importance scores with uncertainty estimates and demonstrates improved explanation reproducibility and stability compared to existing approaches like LIME.
Researchers present the Artificial Agency Program (AAP), a framework for developing AI systems as resource-bounded agents driven by curiosity and learning progress under physical constraints. The program aims to create AI that enhances human capabilities through better sensing, understanding, and action while reducing interface friction between people, tools, and environments.
