29 articles tagged with #model-interpretability. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
AIBullisharXiv – CS AI · 2d ago7/10
🧠FACT-E is a new evaluation framework that uses controlled perturbations to assess the faithfulness of Chain-of-Thought reasoning in large language models, addressing the problem of models generating seemingly coherent explanations with invalid intermediate steps. By measuring both internal chain consistency and answer alignment, FACT-E enables more reliable detection of flawed reasoning and selection of trustworthy reasoning trajectories for in-context learning.
AINeutralarXiv – CS AI · 2d ago7/10
🧠Researchers introduce Pando, a benchmark that evaluates mechanistic interpretability methods by controlling for the 'elicitation confounder'—where black-box prompting alone might explain model behavior without requiring white-box tools. Testing 720 models, they find gradient-based attribution and relevance patching improve accuracy by 3-5% when explanations are absent or misleading, but perform poorly when models provide faithful explanations, suggesting interpretability tools may provide limited value for alignment auditing.
AINeutralarXiv – CS AI · Apr 77/10
🧠A new research study reveals that truth directions in large language models are less universal than previously believed, with significant variations across different model layers, task types, and prompt instructions. The findings show truth directions emerge earlier for factual tasks but later for reasoning tasks, and are heavily influenced by model instructions and task complexity.
AINeutralarXiv – CS AI · Apr 77/10
🧠Researchers identified a sparse routing mechanism in alignment-trained language models where gate attention heads detect content and trigger amplifier heads that boost refusal signals. The study analyzed 9 models from 6 labs and found this routing mechanism distributes at scale while remaining controllable through signal modulation.
AINeutralarXiv – CS AI · Apr 67/10
🧠Researchers analyzed the geometric structure of layer updates in deep language models, finding they decompose into a dominant tokenwise component and a geometrically distinct residual. The study shows that while most updates behave like structured reparameterizations, functionally significant computation occurs in the residual component.
AINeutralarXiv – CS AI · Mar 267/10
🧠Researchers analyzed how large language models (4B-72B parameters) internally represent different ethical frameworks, finding that models create distinct ethical subspaces but with asymmetric transfer patterns between frameworks. The study reveals structural insights into AI ethics processing while highlighting methodological limitations in probing techniques.
AINeutralarXiv – CS AI · Mar 177/10
🧠Researchers introduce Distributional Semantics Tracing (DST), a new framework for explaining hallucinations in large language models by tracking how semantic representations drift across neural network layers. The method reveals that hallucinations occur when models are pulled toward contextually inconsistent concepts based on training correlations rather than actual prompt context.
AIBullisharXiv – CS AI · Mar 46/102
🧠Researchers developed SAE-based Transferability Score (STS), a new metric using sparse autoencoders to predict how well fine-tuned large language models will perform across different domains without requiring actual training. The method achieves correlation coefficients above 0.7 with actual performance changes and provides interpretable insights into model adaptation.
AINeutralarXiv – CS AI · Mar 47/102
🧠Researchers developed linear probes that can predict whether large language models will answer questions correctly by analyzing neural activations before any answer is generated. The method works across different model sizes and generalizes to out-of-distribution datasets, though it struggles with mathematical reasoning tasks.
AINeutralarXiv – CS AI · Mar 46/103
🧠Researchers found that narrow finetuning of Large Language Models leaves detectable traces in model activations that can reveal information about the training domain. The study demonstrates that these biases can be used to understand what data was used for finetuning and suggests mixing pretraining data into finetuning to reduce these traces.
AINeutralarXiv – CS AI · Feb 277/105
🧠Researchers propose Geodesic Integrated Gradients (GIG), a new method for explaining AI model decisions that uses curved paths instead of straight lines to compute feature importance. The method addresses flawed attributions in existing approaches by integrating gradients along geodesic paths under a model-induced Riemannian metric.
AINeutralarXiv – CS AI · 2d ago6/10
🧠Researchers propose a geometric methodology using a Topological Auditor to detect and eliminate shortcut learning in deep neural networks, forcing models to learn fair representations. The approach reduces demographic bias vulnerabilities from 21.18% to 7.66% while operating more efficiently than existing post-hoc debiasing techniques.
AINeutralarXiv – CS AI · 2d ago6/10
🧠Researchers conducted a mechanistic analysis of looped reasoning language models, discovering that these recurrent architectures learn inference stages similar to feedforward models but execute them iteratively. The study reveals that recurrent blocks converge to distinct fixed points with stable attention behavior, providing architectural insights for improving LLM reasoning capabilities.
AINeutralarXiv – CS AI · 2d ago6/10
🧠Researchers propose a novel framework for improving symbolic distillation of neural networks by regularizing teacher models for functional smoothness using Jacobian and Lipschitz penalties. This approach addresses the core challenge that standard neural networks learn complex, irregular functions while symbolic regression models prioritize simplicity, resulting in poor knowledge transfer. Results across 20 datasets demonstrate statistically significant improvements in predictive accuracy for distilled symbolic models.
AINeutralarXiv – CS AI · 6d ago6/10
🧠Researchers discovered that large language models have a fundamental limitation in latent reasoning: they can discover multi-step planning strategies without explicit supervision, but only up to depths of 3-7 steps depending on model size and training method. This finding suggests that complex reasoning tasks may require explicit chain-of-thought monitoring rather than relying on hidden internal computations.
🧠 GPT-4🧠 GPT-5
AIBullisharXiv – CS AI · Apr 76/10
🧠Researchers discovered that multilingual MoE AI models exhibit 'Language Routing Isolation,' where high and low-resource languages activate different expert sets. They developed RISE, a framework that exploits this isolation to improve low-resource language performance by up to 10.85% F1 score while preserving other language capabilities.
AINeutralarXiv – CS AI · Apr 76/10
🧠Researchers conducted the first comprehensive analysis of emotion representations in small language models (100M-10B parameters), finding that these models do possess internal emotion vectors similar to larger frontier models. The study evaluated 9 models across 5 architectural families and discovered that emotion representations localize at middle transformer layers, with generation-based extraction methods proving superior to comprehension-based approaches.
🏢 Perplexity🧠 Llama
AINeutralarXiv – CS AI · Mar 176/10
🧠Researchers introduce Gradient Atoms, an unsupervised method that decomposes AI model training gradients to discover interpretable behaviors without requiring predefined queries. The technique can identify model behaviors like refusal patterns and arithmetic capabilities, while also serving as effective steering vectors to control model outputs.
AINeutralarXiv – CS AI · Mar 176/10
🧠Researchers discovered that transformer language models process factual information through rotational dynamics rather than magnitude changes, actively suppressing incorrect answers instead of passively failing. This geometric pattern only emerges in models above 1.6B parameters, suggesting a phase transition in factual processing capabilities.
AIBullisharXiv – CS AI · Mar 126/10
🧠Researchers introduce CUPID, a plug-in framework that estimates both aleatoric and epistemic uncertainty in deep learning models without requiring model retraining. The modular approach can be inserted into any layer of pretrained networks and provides interpretable uncertainty analysis for high-stakes AI applications.
AINeutralarXiv – CS AI · Mar 96/10
🧠Researchers analyzed Vision-Language Models (VLMs) used in automated driving to understand why they fail on simple visual tasks. They identified two failure modes: perceptual failure where visual information isn't encoded, and cognitive failure where information is present but not properly aligned with language semantics.
AIBullisharXiv – CS AI · Mar 96/10
🧠Researchers developed DEX-AR, a new explainability method for autoregressive Vision-Language Models that generates 2D heatmaps to understand how these AI systems make decisions. The method addresses challenges in interpreting modern VLMs by analyzing token-by-token generation and visual-textual interactions, showing improved performance across multiple benchmarks.
🏢 Perplexity
AINeutralarXiv – CS AI · Mar 66/10
🧠Researchers replicated and extended AI introspection studies, finding that large language models detect injected thoughts through two distinct mechanisms: probability-matching based on prompt anomalies and direct access to internal states. The direct access mechanism is content-agnostic, meaning models can detect anomalies but struggle to identify their semantic content, often confabulating high-frequency concepts.
AIBullisharXiv – CS AI · Mar 36/106
🧠Researchers introduce GlassMol, a new interpretable AI model for molecular property prediction that addresses the black-box problem in drug discovery. The model uses Concept Bottleneck Models with automated concept curation and LLM-guided selection, achieving performance that matches or exceeds traditional black-box models across thirteen benchmarks.
AINeutralarXiv – CS AI · Mar 36/104
🧠Researchers investigated whether large language models can introspect by detecting perturbations to their internal states using Meta-Llama-3.1-8B-Instruct. They found that while binary detection methods from prior work were flawed due to methodological artifacts, models do show partial introspection capabilities, localizing sentence injections at 88% accuracy and discriminating injection strengths at 83% accuracy, but only for early-layer perturbations.
