#model-transparency News & Analysis

32 articles tagged with #model-transparency. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

32 articles

AINeutralarXiv – CS AI · May 286/10

🧠

Singular Vectors of Attention Heads Align with Features

Researchers demonstrate that singular vectors of attention matrices in language models reliably align with learned feature representations, providing theoretical justification for using this mathematical approach to identify interpretable features. The work bridges mechanistic interpretability research by validating why this alignment occurs and proposing testable predictions for detecting it in real models.

AINeutralarXiv – CS AI · May 276/10

🧠

Real-Time Progress Prediction in Reasoning Language Models

Researchers have developed methods to predict real-time progress in reasoning language models with long chains of thought, achieving a 0.161 MAE on mathematical tasks. The work addresses the opacity problem in extended reasoning by training linear probes on hidden states and fine-tuning models to generate percentage-based progress estimates, while quantifying the inherent ambiguity in progress labeling across different model sizes.

AINeutralarXiv – CS AI · Apr 206/10

🧠

Applied Explainability for Large Language Models: A Comparative Study

Researchers compare three explainability techniques—Integrated Gradients, Attention Rollout, and SHAP—for interpreting LLM decisions on sentiment classification tasks. The study reveals that gradient-based methods offer stability and interpretability, while attention-based approaches are faster but less predictive, highlighting critical trade-offs in choosing explanation methods for transformer models.

AINeutralarXiv – CS AI · Apr 146/10

🧠

Latent Structure of Affective Representations in Large Language Models

Researchers investigate how large language models represent emotions in their latent spaces, discovering that LLMs develop coherent emotional representations aligned with established psychological models of valence and arousal. The findings support the linear representation hypothesis used in AI transparency methods and demonstrate practical applications for uncertainty quantification in emotion processing tasks.

AIBullishMIT News – AI · Mar 96/10

🧠

Improving AI models’ ability to explain their predictions

Researchers have developed a new approach to improve AI models' ability to explain their predictions, which could help users determine whether to trust model outputs. This advancement is particularly important for safety-critical applications such as healthcare and autonomous driving where understanding AI decision-making is crucial.

AINeutralOpenAI News · May 285/104

🧠

Teaching models to express their uncertainty in words

The article title suggests coverage of research into teaching AI models to verbally express uncertainty, but no article content was provided for analysis. This represents a significant area of AI development focused on improving model transparency and reliability.

AINeutralLil'Log (Lilian Weng) · Aug 15/10

🧠

How to Explain the Prediction of a Machine Learning Model?

Machine learning models are increasingly being deployed in critical sectors including healthcare, justice systems, and financial services. This necessitates the development of model interpretability methods to understand how AI systems make decisions and ensure compliance with ethical and legal requirements.

← PrevPage 2 of 2