#counterfactual-analysis News & Analysis

6 articles tagged with #counterfactual-analysis. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

6 articles

AINeutralarXiv – CS AI · Jun 17/10

🧠

Counterfactual Evaluation Reveals Hidden Capability Profiles in Clinical LLMs and Agents

Researchers introduce the Causal Sensitivity Score (CSS), an interventional metric that evaluates clinical AI systems by mutating patient case variables to test whether models appropriately adjust recommendations. Testing reveals that six frontier LLMs rank nearly opposite to coverage-based benchmarks, with one model excelling at CSS while performing worst on traditional metrics, exposing a universal safety blind spot where all models fail on surgery-status changes.

AIBearisharXiv – CS AI · Apr 147/10

🧠

Thinking Fast, Thinking Wrong: Intuitiveness Modulates LLM Counterfactual Reasoning in Policy Evaluation

A new study reveals that large language models fail at counterfactual reasoning when policy findings contradict intuitive expectations, despite performing well on obvious cases. The research demonstrates that chain-of-thought prompting paradoxically worsens performance on counter-intuitive scenarios, suggesting current LLMs engage in 'slow talking' rather than genuine deliberative reasoning.

AINeutralarXiv – CS AI · May 126/10

🧠

Investigating Anisotropy in Visual Grounding under Controlled Counterfactual Perturbations

Researchers investigate why visual grounding models fail when image captions are semantically mismatched, hypothesizing that embedding anisotropy may be responsible. Testing two transformer-based models with different embedding geometries reveals no meaningful correlation between cosine similarity and approximation errors, suggesting the problem requires investigation of deeper geometric properties.

AINeutralarXiv – CS AI · May 116/10

🧠

The Translation Tax Is Not a Scalar: A Counterfactual Audit of English-Source Cue Inheritance in Chinese Multilingual Benchmarks

Researchers challenge the assumption that the 'Translation Tax'—a uniform penalty in translated multilingual benchmarks—operates as a simple scalar. Through counterfactual analysis of English-to-Chinese translations, they find translation quality effects are heterogeneous, model-dependent, and item-specific rather than uniform across benchmarks.

AINeutralarXiv – CS AI · Apr 106/10

🧠

CAFP: A Post-Processing Framework for Group Fairness via Counterfactual Model Averaging

Researchers introduce CAFP, a post-processing framework that mitigates algorithmic bias by averaging predictions across factual and counterfactual versions of inputs where sensitive attributes are flipped. The model-agnostic approach eliminates the need for retraining or architectural modifications, making fairness interventions practical for deployed systems in high-stakes domains like credit scoring and criminal justice.

🏢 Meta

AINeutralarXiv – CS AI · Mar 27/1013

🧠

Causal Identification from Counterfactual Data: Completeness and Bounding Results

Researchers developed the CTFIDU+ algorithm for causal identification using counterfactual data, establishing theoretical limits for exact causal inference in non-parametric settings. The work extends previous completeness results by incorporating Layer 3 counterfactual distributions that can be experimentally obtained, and provides novel bounds for non-identifiable quantities.