AINeutralarXiv – CS AI · Jun 197/10
🧠Researchers demonstrate that Large Language Models lack genuine self-awareness regarding their knowledge limitations when applied to clinical tabular data, using cross-model attribution divergence to detect epistemic blind spots. LLM confidence scores remain constant regardless of actual accuracy, while a novel cross-model calibrator achieves reliable uncertainty quantification without model access or retraining.
AIBullisharXiv – CS AI · Jun 197/10
🧠Researchers introduce a prompt-based uncertainty decomposition method that enables LLM agents to proactively seek clarification when task specifications are ambiguous. The approach separates action confidence from request uncertainty and demonstrates 36-73% improvements in clarification performance across multiple LLM backbones compared to existing uncertainty frameworks.
🧠 GPT-5
AIBearisharXiv – CS AI · Jun 117/10
🧠Researchers discover that Chain-of-Thought reasoning in large language models can paradoxically increase overconfidence when reasoning budgets exceed task-specific thresholds, a phenomenon called Calibration Drift Under Reasoning (CDUR). The study shows that while extended reasoning initially improves accuracy, it eventually produces internally consistent but incorrect explanations that mislead models into false confidence, with implications for safe LLM deployment.
🧠 Llama
AIBullisharXiv – CS AI · Jun 107/10
🧠Researchers introduce Program-based Posterior Training (PPT), a novel fine-tuning method that uses probabilistic programs to train LLMs on inductive reasoning tasks. By generating synthetic scenarios and using probabilistic inference to create distributional targets, the approach significantly improves model accuracy on uncertainty estimation while better aligning with human judgment.
AIBullisharXiv – CS AI · Jun 107/10
🧠Researchers propose the first application of split conformal prediction to neural operators for physics simulation, enabling distribution-free uncertainty quantification with formal coverage guarantees. The method achieves 89.1% empirical coverage on heat conduction benchmarks while providing spatially adaptive prediction intervals, addressing a critical gap in deploying AI models for safety-critical engineering applications.
🏢 Nvidia
AIBullisharXiv – CS AI · Jun 107/10
🧠Researchers propose Global-Local Uncertainty (GLU), a new method for quantifying uncertainty in large language models by combining hidden-state geometric entropy with token-level signals. The approach successfully identifies confident-but-wrong predictions that existing token-only methods miss, offering improved reliability assessment across multiple model families.
AIBullisharXiv – CS AI · Jun 97/10
🧠Researchers propose Inference-Time Conformal Reasoning (ITCR), a framework that integrates conformal prediction directly into LLM reasoning generation to provide mathematically valid factuality guarantees. The method addresses the structural nature of uncertainty in multi-step reasoning by calibrating when to stop generation based on graph-level factuality signals, delivering more accurate outputs than post-hoc correction approaches.
AIBullisharXiv – CS AI · Jun 97/10
🧠Researchers introduce FASE (Fast Adaptive Semantic Entropy), a novel metric for evaluating code quality in multi-agent AI systems that reduces computational costs by 99.7% while improving accuracy by 25% compared to existing semantic entropy methods. The approach uses structural and semantic dissimilarity graphs instead of expensive LLM-driven equivalence checks, offering practical uncertainty quantification for autonomous software development.
AIBearisharXiv – CS AI · Jun 97/10
🧠Researchers identify a critical failure mode called Cherry-pick Override (CCO) where large language model judges make unsafe directional commitments when evaluating mixed evidence containing both supporting and refuting claims. The study demonstrates that LLM judges incorrectly return definitive verdicts on over 84% of conflicting-evidence cases instead of acknowledging ambiguity, with panel voting amplifying rather than mitigating this bias.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers present a unified co-design framework for neural network processors that integrates network training, hardware mapping, fabrication, and resource allocation as interoperable blocks. The framework treats uncertainty as an explicit, optimizable resource called Confidence alongside traditional metrics like cost and power, enabling modular improvements without restructuring the entire pipeline.
AIBearisharXiv – CS AI · Jun 17/10
🧠Researchers reveal that vision-language models (VLMs) fail to recognize when spatial questions cannot be reliably answered due to occlusion or perspective ambiguity, instead producing overconfident incorrect responses. The study introduces SpatialUncertain, a benchmark showing that current VLMs achieve only 30% accuracy under occlusion and below 10% under perspective challenges, highlighting a critical gap between answer correctness and epistemic awareness.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce Conf-Gen, a framework that extends conformal prediction—a formal uncertainty quantification method—to generative AI models like LLMs and image generators. The work bridges a gap between established machine learning safety techniques and modern unsupervised AI systems, enabling confidence guarantees on generative outputs across multiple domains.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce Metacognitive Memory Policy Optimization (MMPO), a novel training method that improves how AI language model agents manage memory across long-horizon tasks. The approach uses Belief Entropy—a self-supervised metric measuring uncertainty about task state—to provide fine-grained supervision during memory summarization, enabling agents to maintain 97.1% performance even with 1.75M-token contexts.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers propose the Beta-Bernoulli Calibrator (BBC), a novel method that improves large language model forecasting by converting point estimates into probability distributions using both binary outcomes and aggregated human forecast signals. The approach demonstrates better calibration and accuracy than existing post-hoc methods while leveraging epistemic uncertainty as a more reliable error predictor than verbalized confidence.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers propose COSE, a self-evolution framework for large language models that uses confidence signals to filter noisy self-generated training feedback without external verifiers. The method demonstrates consistent improvements across 19 benchmarks and multiple model sizes (0.6B–4B parameters), achieving state-of-the-art performance in reasoning and mathematics tasks.
🧠 Llama
AIBullisharXiv – CS AI · May 287/10
🧠Researchers introduce Reverse Probing, a novel uncertainty quantification framework designed specifically for clinical LLMs that estimates token-level confidence directly from existing summaries rather than sampling new outputs. The method achieves significant performance improvements on clinical datasets while reducing computational costs, advancing the critical goal of making AI systems safer for healthcare applications.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers demonstrate that uncertainty quantification (UQ) methods can effectively detect errors in LLM-generated code by introducing functional equivalence techniques. While token-probability methods transfer well from NLP, sampling-based approaches fail because traditional semantic models cannot distinguish functionally different code. The proposed functional entropy method outperforms existing approaches across most benchmarks.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers present hybrid neural world models that use machine learning surrogates to accelerate physical dynamics simulations while maintaining accuracy at discontinuities like shocks and contacts. The approach achieves 26-72x speedups over traditional solvers while implicitly learning to identify uncertain regions without explicit training, with an optional fallback mode using classical solvers for high-confidence predictions.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce BaLoRA, a Bayesian extension of Low-Rank Adaptation that improves fine-tuning of large AI models by adding uncertainty quantification while narrowing the accuracy gap with full fine-tuning. The method uses input-adaptive parameterization with minimal computational overhead and demonstrates stronger performance across language, vision, and materials science tasks.
AIBullisharXiv – CS AI · May 117/10
🧠Researchers propose a novel uncertainty quantification method for Prior-Data Fitted Networks (PFNs), emerging foundation models for tabular data prediction, using martingale posteriors to provide calibrated confidence estimates. The technique is tuning-free, computationally efficient, and mathematically proven to converge, addressing a significant limitation in PFNs' practical applicability.
AINeutralarXiv – CS AI · May 117/10
🧠Researchers have developed a method to predict whether language model reasoning traces produce correct answers by analyzing uncertainty profiles—patterns in model confidence across generated token sequences. The approach achieves 80.7% accuracy in detecting errors and can identify failures within the first few hundred tokens, providing insights into how LLMs actually perform reasoning tasks.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers introduce BeliefMem, a novel memory architecture for LLM agents that retains multiple candidate conclusions with associated probabilities instead of committing to single deterministic interpretations. This probabilistic approach preserves uncertainty, allows agents to update confidence as new evidence arrives, and demonstrates superior performance on LoCoMo and ALFWorld benchmarks compared to existing memory methods.
AIBullisharXiv – CS AI · May 77/10
🧠Researchers introduce SemGrad, a gradient-based uncertainty quantification method for large language models that operates in semantic space rather than parameter space, eliminating the computational overhead of sampling-based approaches. The method measures output stability under semantically equivalent input perturbations to gauge LLM confidence, addressing the critical challenge of hallucinations in free-form text generation.
AIBullisharXiv – CS AI · Apr 147/10
🧠Researchers demonstrate that variational Bayesian methods significantly improve Vision Language Models' reliability for Visual Question Answering tasks by enabling selective prediction with reduced hallucinations and overconfidence. The proposed Variational VQA approach shows particular strength at low error tolerances and offers a practical path to making large multimodal models safer without proportional computational costs.
AIBullisharXiv – CS AI · Apr 77/10
🧠Researchers developed an LLM-powered evolutionary search method to automatically design uncertainty quantification systems for large language models, achieving up to 6.7% improvement in performance over manual designs. The study found that different AI models employ distinct evolutionary strategies, with some favoring complex linear estimators while others prefer simpler positional weighting approaches.
🧠 Claude🧠 Sonnet🧠 Opus
