AINeutralarXiv – CS AI · Jun 97/10
🧠Researchers present SENTRY, a statistical fault injection framework that efficiently evaluates Vision Transformers' reliability against soft errors in safety-critical applications. The method achieves formal reliability guarantees using finite-population sampling theory, reducing experimental costs by up to 10,700x while identifying critical vulnerabilities in normalization layers and IEEE-754 exponent bits.
AIBearisharXiv – CS AI · Jun 17/10
🧠Researchers demonstrate that mechanistic interpretability—the process of reverse-engineering AI model behaviors through circuit discovery—suffers from fundamental statistical instability due to high variance in causal mediation analysis. The findings reveal that circuit structures are fragile and highly sensitive to input data and hyperparameter changes, calling into question the scientific validity of existing MI methodologies and necessitating stricter statistical practices in the field.
AINeutralarXiv – CS AI · May 287/10
🧠Researchers challenge the GSM-Symbolic benchmark's conclusions about LLM reasoning capabilities, finding that statistical rigor reveals only half of tested models show significant performance degradation. The analysis uncovers a previously unacknowledged distributional shift in problem integers and identifies distinct, model-specific failure patterns rather than universal reasoning deficits.
AINeutralarXiv – CS AI · Apr 147/10
🧠A comprehensive study analyzing 935 ablation experiments from 161 publications reveals that artificial intelligence architectural evolution follows the same statistical laws as biological evolution, with a heavy-tailed distribution of fitness effects placing AI between viral genomes and simple organisms. The findings suggest that evolutionary statistical structure is substrate-independent and determined by fitness landscape topology rather than the underlying selection mechanism.
AINeutralarXiv – CS AI · Mar 117/10
🧠A research study reveals that AI-powered search engines like Perplexity, SearchGPT, and Google Gemini produce highly variable citation results for identical queries, making single-run visibility metrics unreliable. The study demonstrates that citation distributions follow power-law patterns with substantial variability, and argues that uncertainty estimates are essential for accurate measurement of domain visibility in generative search.
🏢 OpenAI🏢 Perplexity🧠 Gemini
AINeutralarXiv – CS AI · Jun 236/10
🧠Researchers introduced StatABench, a comprehensive benchmark for evaluating LLMs' statistical analysis capabilities across 434 questions and tasks. Evaluations reveal significant performance gaps, with GPT-5.1 achieving only 68.6% accuracy on closed-ended questions and top agent frameworks scoring 61.86% on complex modeling tasks, exposing persistent weaknesses in tool-grounded reasoning and methodological decision-making.
🧠 GPT-5
AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers present a new statistical framework for evaluating generative models by estimating Precision-Recall curves through a binary classification approach. The work provides theoretical guarantees including minimax upper bounds on estimation risk and unifies several existing PR metrics under a single framework.
AINeutralarXiv – CS AI · May 96/10
🧠Researchers introduce PKeX-Shapley, an algorithm that computes exact Shapley values for product-kernel machine learning models in quadratic time, eliminating the need for approximations. The method exploits the multiplicative structure of product kernels to achieve linear-time-per-feature attribution without sampling or density estimation, extending beyond predictive models to statistical discrepancy measures like MMD and HSIC.
AINeutralarXiv – CS AI · Mar 24/106
🧠Researchers demonstrate that Independent Component Analysis (ICA) can be effectively used for treatment effect estimation by exploiting the same moment conditions as higher-order Orthogonal Machine Learning. The study proves linear ICA can consistently estimate multiple treatment effects and shows sample-efficiency advantages over OML in certain scenarios.
