69 articles tagged with #explainability. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers find that large language models make decisions based on systematic behavioral patterns but struggle to accurately articulate their reasoning. The study reveals a disconnect between what LLMs claim influences their choices and the attributes that actually drive their decisions, suggesting models operate with 'superficial beliefs' rather than fully understood decision frameworks.
Researchers introduce Visual-TCAV, a novel explainability framework for image classification that combines concept-based and saliency-based methods to provide both local and global interpretations of CNN predictions. The method demonstrates improved faithfulness compared to existing approaches like TCAV, bridging a gap between understanding where networks recognize concepts and how those concepts contribute to specific predictions.
Researchers have developed a novel framework extending Shapley Values—a traditional explainability method—to multimodal large language models that process both text and audio. The work introduces computational optimizations and a preprocessing technique called Spectrogram-Guided Phonetic Alignment to make the analysis feasible, alongside an open-source tool for visualization, revealing that input modality significantly affects model attribution patterns.
Researchers introduce CEF-Log, an LLM-based method for detecting malicious web server logs that achieves 99% F1-score using only four examples while generating forensically explainable reasoning. The approach embeds investigative methodology through structured chain-of-thought prompting, addressing the critical need for both accuracy and legal-admissible explanations in cybersecurity forensics.
Researchers introduce the Governance-Aware Autonomous Testing Framework (GATF), which adds governance validation, compliance monitoring, and explainability controls to AI-powered software testing systems. The framework achieved 89.6% reduction in governance-related risks and demonstrated high accuracy across multiple performance metrics, addressing critical concerns about AI-generated test artifacts including hallucinations and security vulnerabilities.
Researchers demonstrate that Concept Bottleneck Models and Sparse Autoencoders, two distinct interpretability approaches in machine learning, share an underlying geometric structure based on concept cones. This unification enables quantitative evaluation of how well unsupervised concept discovery aligns with human-defined concepts, advancing AI interpretability standards.
Researchers propose the Glassbox Framework, a new AI architecture that replaces post-hoc explainability with ante-hoc probabilistic mediation using Bayesian networks as transparent reasoning layers for large language models. This approach aims to make AI systems fundamentally accountable in high-stakes domains like healthcare, law, and public administration by encoding domain knowledge and causal assumptions before inference occurs.
Researchers present an abstract architecture for building autonomous robotic systems that can explain their decision-making processes to human operators and regulators. The framework addresses the critical need for explainability in autonomous systems deployed in hazardous environments, with a practical application example in nuclear industry operations where trust and regulatory compliance are essential.
Researchers introduce TRUE (Trustworthy Unified Explanation Framework), a new methodology for interpreting and verifying the reasoning processes of large language models across multiple analytical levels. The framework combines executable verification, structural analysis, and causal failure mode detection to provide transparent insights into LLM decision-making, addressing critical gaps in current interpretability methods.
Researchers propose a framework combining SHAP explainability with LLM-generated rationales to improve transparency in automated rubric-based scoring systems for educational assessment. Testing on classroom transcripts reveals fine-tuned language models outperform LLMs in accuracy, but SHAP attributions provide more faithful and transferable explanations than LLM rationales across different model architectures.
Researchers propose a metamorphic testing framework to evaluate the trustworthiness of machine learning model explanations by identifying inconsistencies between model predictions and feature attributions, addressing the Rashomon effect where multiple models achieve similar performance but yield conflicting explanations.
A comprehensive survey examines evidence tracing and execution provenance in LLM agents—mechanisms for tracking how autonomous AI systems arrive at decisions by documenting retrieved evidence, tool interactions, and memory influences. This research addresses critical gaps in verifying, debugging, and auditing agent behavior beyond simple output accuracy, proposing frameworks and taxonomies for process-level accountability in AI systems.
Researchers propose AttnRegDeepLab, a deep learning framework that automates embryo fragmentation grading for IVF procedures with improved clinical interpretability. The method combines attention-guided segmentation with regression analysis to eliminate subjective manual assessment while maintaining accuracy and transparency in developmental potential evaluation.
Researchers propose PEACE, a planner-executor agent architecture for autonomous drones that decouples high-level mission planning from low-level control using foundation models. The system combines large language models for task planning with structured tool-calling interfaces and constraint enforcement mechanisms, demonstrating improved explainability and reduced computational overhead compared to tightly coupled LLM approaches.
Researchers propose GCAN, a novel deep learning framework that uses counterfactual generation and brain atlas constraints to improve the explainability of cognitive decline diagnosis from brain imaging data. The method achieves competitive classification performance on mild cognitive impairment and subjective cognitive decline detection while providing interpretable insights into disease-related connectivity changes.
Researchers developed EviOSAHS, an evidence-grounded AI framework that combines visual analysis of facial features with clinical data to screen for obstructive sleep apnea, achieving 94.86% sensitivity and outperforming direct multimodal prompting approaches. The system decomposes facial images into seven anatomical queries before final clinical adjudication, providing a more reliable and auditable screening workflow than traditional foundation model prompting.
Researchers have developed an alternative to deep neural networks for large language models based on RBF (Radial Basis Function) networks that claims to find optimal solutions in closed form without iterative training. The approach promises improved explainability and accuracy while eliminating the computationally expensive training process required by traditional DNNs.
Researchers introduce Xetrieval, a mechanistic framework that explains how dense retrieval models assign relevance scores by decomposing high-dimensional embeddings into interpretable features. The method uses a lightweight reasoning internalizer to enrich embeddings with reasoning information and provides human-readable feature-level explanations of retrieval decisions, advancing transparency in neural information retrieval systems.
Researchers introduce eXTC, a new framework combining structured prompt optimization with reinforcement learning to create interpretable text classifiers that balance performance with explainability. The system generates human-readable domain rules while maintaining inference speed through knowledge distillation, addressing a longstanding trade-off in AI transparency.
Researchers introduce SafeRx-Agent, a multi-agent AI framework designed to improve medication recommendation systems by integrating clinical knowledge, safety verification, and explainability. The system addresses limitations in existing approaches by using fine-grained drug classification (ATC codes) and demonstrating improved accuracy while controlling for drug interactions and contraindications on MIMIC datasets.
Researchers have developed TELL, an AI-generated text detector that prioritizes explainability by showing users the specific linguistic markers indicating AI or human authorship rather than just providing an opaque numerical score. The system achieves competitive detection performance (AUROC 0.927) while generating human-evaluated explanations with a 72.3% mean win-rate across quality metrics, fundamentally reframing detection as a human-centric interpretability problem.
Researchers found that large language models' chain-of-thought reasoning remains remarkably consistent even when reaching opposite conclusions about conflicting information, suggesting CoT explanations don't faithfully reflect the underlying decision mechanism. While model confidence shows weak but genuine predictive signal for decisions, internal reasoning tokens proved more decision-sensitive than user-facing explanations, indicating models may not transparently report how they actually choose between document claims and training knowledge.
Researchers identify a significant gap between evolutionary computation research and real-world physics-based optimization applications. Domain experts consistently require fast convergence and algorithm explainability, but existing evolutionary algorithm techniques remain underutilized in complex practical scenarios due to trust and performance concerns.
Researchers have developed READER, a compact AI text detector with only 1.5B parameters that outperforms much larger language models and existing detection systems. READER combines classification with explainable reasoning, providing both AI/human verdicts and structured rationales for its decisions, addressing critical limitations in current detection methods that fail under distribution shifts.
Researchers have developed attribution techniques that explain decision-making in Markov Decision Processes (MDPs), extending explainability methods beyond static inputs to sequential decision-making systems. The approach assigns importance scores to states and execution paths, enabling more interpretable AI agents in dynamic environments.
