AINeutralarXiv – CS AI · Jun 237/10
🧠Researchers have developed Trust Elasticity (TE), a metric measuring how readily large language models change their outputs when presented with conflicting evidence. The study finds that internal uncertainty indicators—such as confidence miscalibration—correlate with behavioral variation in how different LLMs resolve cognitive dissonance, suggesting future AI safety interventions could target these measurable internal properties.
🧠 Llama
AIBearisharXiv – CS AI · Jun 237/10
🧠Researchers reveal that multimodal language models used as judges fail to fairly evaluate culturally ambiguous content, exhibiting calibration and orientation biases when assessed against diverse human annotators. The study demonstrates these models systematically favor one cultural perspective while compressing their scoring scales, with implications for any AI system deployed across cultural contexts.
AINeutralarXiv – CS AI · Jun 197/10
🧠Researchers demonstrate that Large Language Models lack genuine self-awareness regarding their knowledge limitations when applied to clinical tabular data, using cross-model attribution divergence to detect epistemic blind spots. LLM confidence scores remain constant regardless of actual accuracy, while a novel cross-model calibrator achieves reliable uncertainty quantification without model access or retraining.
AIBullisharXiv – CS AI · Jun 107/10
🧠Researchers introduce Program-based Posterior Training (PPT), a novel fine-tuning method that uses probabilistic programs to train LLMs on inductive reasoning tasks. By generating synthetic scenarios and using probabilistic inference to create distributional targets, the approach significantly improves model accuracy on uncertainty estimation while better aligning with human judgment.
AINeutralarXiv – CS AI · May 127/10
🧠Researchers identify a critical vulnerability in agentic memory systems where Large Language Models retrieve and amplify spurious correlations from stored information, leading to erroneous reasoning in downstream decisions. The study benchmarks this risk and proposes CAMEL, a lightweight calibration method that mitigates spurious pattern reliance while maintaining performance on clean data.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers propose a novel uncertainty quantification method for Prior-Data Fitted Networks (PFNs), emerging foundation models for tabular data prediction, using martingale posteriors to provide calibrated confidence estimates. The technique is tuning-free, computationally efficient, and mathematically proven to converge, addressing a significant limitation in PFNs' practical applicability.
AIBullisharXiv – CS AI · Apr 147/10
🧠Researchers demonstrate that variational Bayesian methods significantly improve Vision Language Models' reliability for Visual Question Answering tasks by enabling selective prediction with reduced hallucinations and overconfidence. The proposed Variational VQA approach shows particular strength at low error tolerances and offers a practical path to making large multimodal models safer without proportional computational costs.
AIBullisharXiv – CS AI · Apr 137/10
🧠Researchers propose Evidential Transformation Network (ETN), a lightweight post-hoc module that converts pretrained models into evidential models for uncertainty estimation without retraining. ETN operates in logit space using sample-dependent affine transformations and Dirichlet distributions, demonstrating improved uncertainty quantification across vision and language benchmarks with minimal computational overhead.
AINeutralarXiv – CS AI · Jun 256/10
🧠Researchers released Argus, a comprehensive benchmark for uncertainty quantification in AI agents that control computers through GUI interactions. The study evaluated 27 uncertainty methods across multiple vision-language models and datasets, finding that uncertainty rankings remain stable within a single model but degrade significantly when switching between different model classes or interfaces.
AIBullisharXiv – CS AI · Jun 256/10
🧠Researchers demonstrate that Sharpness-Aware Minimization (SAM), a recently proposed neural network training method, significantly improves model calibration by reducing overconfidence in predictions. The study includes a new variant called CSAM that further enhances calibration performance across multiple datasets, with important implications for safety-critical AI applications.
AIBullisharXiv – CS AI · Jun 106/10
🧠Researchers propose a density ridge-based method for detecting hallucinations in large language and vision-language models that outperforms existing approaches by 5-20 AUROC points while requiring minimal calibration labels. The technique maps hidden state trajectories to a low-dimensional geometric skeleton, enabling robust hallucination detection even when training data is scarce.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers propose TRACE, a novel machine unlearning technique designed specifically for Mixture-of-Experts language models that addresses the problem of forget-critical experts receiving insufficient regularization during the unlearning process. The method achieves 9% relative utility improvements by detecting and calibrating expert activation patterns to match forget and retain data distributions, demonstrating consistent performance gains across multiple MoE architectures.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers propose a hybrid machine learning architecture combining FT-Transformer neural networks with XGBoost gradient boosting to predict customer churn in banking and subscription services. The ensemble method achieves superior performance metrics (62.10% F1, 0.861 AUC-ROC) compared to baseline models while addressing critical challenges in class imbalance and probability calibration.
AINeutralarXiv – CS AI · Jun 96/10
🧠Researchers introduce TRIAGE, an LLM-based framework that uses dialectical reasoning to improve risk prediction on irregularly sampled medical time series data. The approach generates competing clinical outcome rationales to produce calibrated, continuous risk scores rather than overconfident binary predictions, achieving 3.3% AUPRC improvement and 81% reduction in calibration error versus baseline methods.
AIBullisharXiv – CS AI · Jun 96/10
🧠Researchers propose a meta-cognitive framework that improves Large Language Models by distinguishing between mastered knowledge, confused understanding, and missing information. The approach uses internal confidence signals to guide targeted knowledge augmentation and calibrate model certainty with actual accuracy, addressing a critical gap where LLMs often exhibit overconfidence despite knowledge deficiencies.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers present a method to improve error prediction in Large Language Models by distinguishing between genuine model uncertainty and input ambiguity. Using uncertainty quantification metrics on question-answering tasks, they demonstrate that ambiguity information significantly enhances error prediction accuracy, yielding improvements exceeding 10 percentage points across multiple datasets and model families.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers propose Sequential Bayesian Belief Tracking (SBBT), a framework for estimating the reliability of long reasoning chains in large language models before final answers are known. The study finds that probability calibration and ranking performance respond differently to various evidence types: scalar scores improve calibration metrics, while structural observations are needed for ranking tasks.
AINeutralarXiv – CS AI · May 286/10
🧠Researchers introduce EvaluatorDPT, a decision-control model that predicts YES, NO, or TBD (to-be-determined) for high-stakes AI applications where uncertainty exists. The system learns deferral as an explicit outcome rather than hiding uncertainty in forced predictions, achieving 82.6% accuracy with auditable, policy-governed decision routing that can be inspected and controlled at inference time.
AINeutralarXiv – CS AI · May 285/10
🧠Researchers propose Under-Cali, a machine learning framework for forecasting irregular multivariate time series data in real-time online settings. The system uses uncertainty estimation and dual-expert calibration to maintain accuracy despite dynamic data distribution shifts, achieving improvements over existing methods with minimal computational overhead.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers introduce the Metacognitive Probe, a diagnostic tool measuring five dimensions of LLM confidence behavior including calibration, epistemic vigilance, and reasoning validation. Testing on eight frontier models and 69 humans reveals significant within-model disparities—exemplified by Gemini 2.5 Flash scoring 88 on confidence calibration but only 41 on difficulty prediction—suggesting composite benchmarks mask pockets of overconfidence.
🧠 Gemini
AINeutralarXiv – CS AI · May 126/10
🧠NoisyCoconut is an inference-time method that improves LLM reliability by injecting controlled noise into internal representations to generate diverse reasoning paths, enabling models to abstain when uncertain without requiring retraining. The technique reduces error rates from 40-70% to below 15% on mathematical reasoning tasks through unanimous agreement among noise-perturbed paths, offering practical reliability improvements compatible with existing models.
AINeutralarXiv – CS AI · May 116/10
🧠Researchers propose using multidimensional self-assessment based on cognitive appraisal theory to predict LLM failures more reliably than confidence alone. Testing across 12 models and 38 tasks, they find effort and ability dimensions consistently outperform confidence, with task type determining which dimension proves most predictive.
AINeutralarXiv – CS AI · May 76/10
🧠Researchers propose Adaptive Conformal Semantic Entropy (ACSE), a novel method for quantifying uncertainty in large language model outputs by measuring semantic diversity rather than relying solely on lexical or probabilistic measures. The approach uses conformal calibration to provide statistical guarantees on error rates, demonstrating significant performance improvements over existing uncertainty quantification baselines.
AINeutralarXiv – CS AI · Apr 136/10
🧠Researchers introduce VOLTA, a simplified deep learning approach for uncertainty quantification that outperforms ten established baselines including ensemble methods and MC Dropout. The method achieves superior calibration with expected calibration error of 0.010 and competitive accuracy across multiple datasets, suggesting that complex auxiliary losses may be unnecessary for reliable uncertainty estimation in safety-critical applications.
AIBullisharXiv – CS AI · Apr 106/10
🧠Researchers demonstrate that Large Language Models used as judges suffer from score range bias, where evaluation outputs are highly sensitive to predefined scoring scales. Using contrastive decoding techniques, they achieve up to 11.7% improvement in alignment with human judgments across different score ranges.
