y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#interpretability News & Analysis

366 articles tagged with #interpretability. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

366 articles
AIBearisharXiv – CS AI · Jun 47/10
🧠

Lost in Fog: Sensor Perturbations Expose Reasoning Fragility in Driving VLAs

Researchers evaluated Vision-Language-Action models in autonomous driving under sensor degradation, finding that explanation consistency (Chain-of-Causation) strongly correlates with trajectory reliability. When model explanations change due to perturbations like fog or noise, trajectory errors increase 5.3x, suggesting reasoning consistency could serve as a safety monitoring tool for autonomous vehicles.

AIBullisharXiv – CS AI · Jun 27/10
🧠

Multimodal Function Vectors for Visual Relations

Researchers demonstrate that Large Multimodal Models encode visual relational knowledge in specific attention heads called function vectors, which can be extracted and manipulated to improve performance on relational tasks. These vectors can be fine-tuned with minimal data while keeping model parameters frozen, and can be linearly combined to solve novel analogy problems, advancing understanding of how multimodal AI systems process visual relationships.

AINeutralarXiv – CS AI · Jun 27/10
🧠

Visual Persuasion: What Influences Decisions of Vision-Language Models?

Researchers developed a framework to systematically study how vision-language models (VLMs) make visual decisions by perturbing images and measuring preference shifts. Using visual prompt optimization techniques, they identified consistent visual themes that influence VLM choices, revealing potential safety vulnerabilities in image-based AI agents operating at scale.

AIBullisharXiv – CS AI · Jun 27/10
🧠

A Monosemantic Attribution Framework for Stable Interpretability in Clinical Neuroscience Transformer-Based Language Models

Researchers have developed a monosemantic attribution framework to improve interpretability of Transformer-based language models in clinical applications, particularly for Alzheimer's disease diagnosis. The framework addresses instability in existing attribution methods by reducing inter-method variability and providing stable, explicit importance scores for model predictions.

AIBullisharXiv – CS AI · Jun 27/10
🧠

Expected Value Alignment for Generative Reward Modeling in Formal Mathematics Verification

Researchers introduce Expected Value Alignment (EVA), a novel reward-modeling technique that enables Large Language Models to provide continuous numerical scores while maintaining human-readable text output for formal mathematics verification in Lean 4. The method bridges a critical gap between discrete generative outputs and continuous value assessment needed for reinforcement learning in theorem proving systems.

AINeutralarXiv – CS AI · Jun 27/10
🧠

Fundamental Limitation in Explaining AI

Researchers have mathematically proven a fundamental theoretical constraint on AI explainability, demonstrating that AI systems cannot simultaneously satisfy four desirable conditions: environmental complexity, performance quality, interpretability, and complete faithfulness of explanations. This finding suggests AI governance frameworks must accept inherent limitations in explanation completeness rather than pursue unattainable perfect transparency.

AINeutralarXiv – CS AI · Jun 27/10
🧠

Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing

Mechanistic interpretability (MI) research lacks standardized auditing systems, causing conflicting findings and limiting adoption in safety-critical applications like medical AI and autonomous systems. Researchers propose a collaborative reviewing platform with continuous feedback, expert-verified guidelines, and source-based auditing to improve the field's credibility and enable broader deployment.

AINeutralarXiv – CS AI · Jun 27/10
🧠

MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

Researchers introduce MENTIS, a framework for measuring internal geometric changes in language models during preference alignment training. The study reveals that alignment leaves selective, depth-localized signatures in model computations, with normative concepts showing larger internal reorganization than factual concepts across multiple model architectures.

AIBullisharXiv – CS AI · Jun 27/10
🧠

Prototype Transformer: Towards Language Model Architectures Interpretable by Design

Researchers introduce Prototype Transformer (ProtoT), a new language model architecture that replaces standard self-attention with a linear-cost prototype-based module to improve interpretability. The approach enables models to automatically learn and represent named concepts, addressing long-standing concerns about opacity in large language models while maintaining competitive performance on standard benchmarks.

AINeutralarXiv – CS AI · May 297/10
🧠

The Hamilton-Jacobi Theory of Deep Learning

Researchers establish a mathematical framework connecting neural network training to Hamilton-Jacobi partial differential equations, showing that gradient descent searches through solutions to viscous PDEs. This theoretical unification applies across major architectures including residual networks and transformers, with implications for understanding generalization, adversarial robustness, and interpretability.

AIBullisharXiv – CS AI · May 297/10
🧠

ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows

ProtoMedAgent introduces a framework that combines interpretable prototype networks with privacy-aware AI workflows to generate clinically accurate medical reports without the hallucination issues common in standard RAG systems. The approach achieves 91.2% faithfulness in clinical documentation while protecting patient privacy through k-anonymity and ℓ-diversity constraints.

AIBullisharXiv – CS AI · May 297/10
🧠

ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology

ConceptM³oE introduces a novel AI architecture that combines multimodal mixture-of-experts with interpretable concept bottlenecks for computational pathology, enabling medical AI models to provide transparent reasoning while maintaining competitive performance. The framework improves diagnostic accuracy in data-limited scenarios and demonstrates practical alignment with clinical decision-making processes.

AIBearisharXiv – CS AI · May 297/10
🧠

When and How Long? The Readout-Mediator Angle in Temporal Reasoning

Researchers demonstrate that linear probes can successfully decode information from neural networks while remaining completely disconnected from how models actually process that information. Using calendar-date reasoning tasks, they show that probes identifying day-of-year information are orthogonal to the causal mechanisms models use for duration reasoning, revealing a fundamental flaw in probe-based interpretability methods.

AIBullisharXiv – CS AI · May 297/10
🧠

Modeling Hierarchical Thinking in Large Reasoning Models

Researchers propose modeling Large Reasoning Models' Chain-of-Thought processes as trajectories through a six-state Finite State Machine, enabling better understanding and control of reasoning dynamics. They introduce Q-Value guided steering, a training-free method that optimizes reasoning by applying sparse activation steering at sentence boundaries, achieving significant performance gains across multiple benchmarks with minimal computational overhead.

AIBullisharXiv – CS AI · May 287/10
🧠

DecomposeRL: Learning to Ask Useful, Informative, and Diverse Questions for Semi-Supervised, Traceable Claim Verification

DecomposeRL presents a novel reinforcement learning approach to claim verification that achieves high accuracy while maintaining interpretability through decomposition-based reasoning. A 7B parameter model trained on just 5K curated claims matches 32B baselines and GPT-4.1-mini across 11 benchmarks while enabling semi-supervised learning, demonstrating efficient scaling through intelligent data curation.

🧠 GPT-4
AIBullisharXiv – CS AI · May 287/10
🧠

VITAL: Visual-Semantic Dual Supervision for Enhanced and Interpretable Latent Reasoning in Medical MLLMs

Researchers introduce VITAL, a latent-space reasoning framework for medical AI models that uses dual visual-semantic supervision to improve medical visual question answering while maintaining interpretability. The method addresses modality collapse and inference efficiency issues in existing approaches, achieving state-of-the-art results on 7 benchmarks using a newly constructed 61K medical imaging dataset.

AIBullisharXiv – CS AI · May 287/10
🧠

Clinical Validation of the Melanoscope AI Mobile Dermoscopy Clinical Decision Support System

Researchers validated the Melanoscope AI clinical decision support system for skin lesion screening in Russian outpatient settings, achieving 88.6% agreement with expert assessment and zero false negatives among malignant cases. The study introduces quantitative interpretability methods for deep learning models and a three-zone patient routing algorithm, demonstrating the viability of AI-powered dermoscopy as a scalable solution to address dermatologist shortages.

AIBearisharXiv – CS AI · May 287/10
🧠

The Attentional White Bear Effect in Transformer Language Models

Researchers discovered that instruction-based suppression in transformer language models fails to eliminate prohibited concepts from internal representations, despite successfully preventing their explicit expression. The study reveals that suppressed content remains recoverable from hidden layers and continues influencing model behavior, exposing a critical gap between behavioral safety and true representational alignment.

AIBullisharXiv – CS AI · May 287/10
🧠

CORE: Contrastive Reflection Enables Rapid Improvements in Reasoning

Researchers introduce CORE (Contrastive Reflection), a non-parametric learning algorithm that improves language model reasoning by comparing successful and unsuccessful problem attempts to generate natural-language insights. The method achieves faster improvements than existing parametric and non-parametric approaches while requiring significantly fewer model rollouts and training samples, offering a more efficient and interpretable alternative to weight updates or prompt optimization.

AIBullisharXiv – CS AI · May 287/10
🧠

Reverse Probing: Supervised Token-level Uncertainty Quantification for Large Language Models in Clinical Text

Researchers introduce Reverse Probing, a novel uncertainty quantification framework designed specifically for clinical LLMs that estimates token-level confidence directly from existing summaries rather than sampling new outputs. The method achieves significant performance improvements on clinical datasets while reducing computational costs, advancing the critical goal of making AI systems safer for healthcare applications.

AIBullisharXiv – CS AI · May 287/10
🧠

OmniVerifier-M1: Multimodal Meta-Verifier with Explicit Structured Recalibration

Researchers introduce OmniVerifier-M1, a multimodal verification system that uses symbolic outputs like bounding boxes rather than text explanations to improve error detection in visual AI models. The approach combines meta-verification feedback with decoupled reinforcement learning to enable more reliable and interpretable verification of multimodal foundation models, with applications in autonomous error correction.

AINeutralarXiv – CS AI · May 287/10
🧠

Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness

Researchers propose Faithful Agentic XAI (FAX), a framework that improves the reliability of AI explanations generated by large language models through explicit verification mechanisms. The study introduces CRAFTER-XAI-Bench, a new benchmark for testing explanation faithfulness in complex environments, demonstrating that current XAI systems can produce plausible but inaccurate explanations that mislead users.

AIBullisharXiv – CS AI · May 287/10
🧠

Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values

Researchers introduce ShaQ, a Shapley-value-based framework that identifies which specific parts of user input cause uncertainty in large language models, rather than just flagging overall uncertainty. The method achieves state-of-the-art ambiguity detection on multiple benchmarks and demonstrates practical value in high-stakes domains like clinical settings by enabling targeted input clarification.

AINeutralarXiv – CS AI · May 287/10
🧠

Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNN

Researchers reverse-engineered a Sokoban-playing RNN trained with model-free reinforcement learning and discovered that the network encodes planning strategies through specialized neural channels that represent directional movements and learned transition models. The findings demonstrate that neural networks can develop interpretable planning algorithms without explicit supervision, with path channels and extension kernels working together to implement bidirectional search and backtracking.

AIBullisharXiv – CS AI · May 277/10
🧠

MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentation

MedVol-R1 introduces a reinforcement learning framework for volumetric reasoning segmentation in 3D medical imaging, decoupling evidence grounding from mask generation to improve interpretability and accuracy. The system uses an LVLM to identify key 2D evidence anchors before propagating them into coherent 3D segmentations, achieving state-of-the-art results on multiple medical imaging benchmarks without requiring expensive annotations.

← PrevPage 2 of 15Next →