AIBullisharXiv – CS AI · 3d ago7/10
🧠Researchers introduce COFT, a training-free decoding method that reduces bias in large language models' chain-of-thought reasoning by 30-55% through counterfactual prompting and conformal calibration. The approach preserves task performance while adding minimal computational overhead, offering a practical solution for deploying fairer AI systems without model retraining.
🏢 Meta
AIBullisharXiv – CS AI · 6d ago7/10
🧠Researchers introduce Conf-Gen, a framework that extends conformal prediction—a formal uncertainty quantification method—to generative AI models like LLMs and image generators. The work bridges a gap between established machine learning safety techniques and modern unsupervised AI systems, enabling confidence guarantees on generative outputs across multiple domains.
AINeutralarXiv – CS AI · Mar 177/10
🧠Researchers propose group-conditional federated conformal prediction (GC-FCP), a new protocol that enables trustworthy AI uncertainty quantification across distributed clients while providing coverage guarantees for specific groups. The framework addresses challenges in federated learning for applications in healthcare, finance, and mobile sensing by creating compact weighted summaries that support efficient calibration.
AINeutralarXiv – CS AI · Feb 277/105
🧠Researchers propose FedWQ-CP, a new approach for uncertainty quantification in federated learning that addresses both data and model heterogeneity challenges. The method enables reliable uncertainty estimation across distributed agents while maintaining efficiency through single-round communication and weighted threshold aggregation.
AINeutralarXiv – CS AI · 2d ago6/10
🧠Researchers benchmark 12 LLMs under compression to evaluate whether quantization and pruning preserve uncertainty quantification alongside accuracy. The study reveals compression frequently decouples accuracy from uncertainty reliability, with smaller models absorbing compression-induced uncertainty poorly, suggesting current accuracy-only evaluation standards are insufficient for deployment readiness.
AINeutralarXiv – CS AI · 2d ago6/10
🧠Researchers propose personalized federated weighted conformal prediction (PFWCP), a framework that enables reliable uncertainty quantification across multiple agents while preserving privacy and handling data heterogeneity. The method provides statistical validity guarantees for individual participants rather than only aggregate averages, with practical applications in distributed machine learning systems.
AIBullisharXiv – CS AI · 2d ago6/10
🧠Researchers propose a new method to certify the safety of belief-space safety filters (BeliefSF) in interactive robotics using conformal prediction, addressing the challenge of providing formal safety guarantees when robots deploy neural approximations and runtime inference. The approach reduces conservativeness in safety filtering while maintaining high-probability safety assurances, demonstrated through human-vehicle interaction simulations.
AIBullisharXiv – CS AI · 2d ago6/10
🧠Researchers propose Domain-Shift-Aware Conformal Prediction (DS-CP), a framework that improves reliability of large language model outputs by adapting conformal prediction methods to handle domain shift. The approach reweights calibration samples based on proximity to test prompts, delivering more reliable uncertainty quantification and reducing hallucinations in real-world deployments.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers introduce Excess Risk of Target Coverage (ERT), a new metric framework for evaluating conditional coverage in conformal prediction systems. The approach reformulates coverage assessment as a classification problem, providing more statistically powerful diagnostics than existing methods while offering conservative estimates of miscoverage and enabling distinction between over- and under-coverage effects.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers introduce SCOPE, a framework that improves LLM-based pairwise evaluation by calibrating confidence thresholds to control error rates. Combined with a new uncertainty metric called Bidirectional Preference Entropy (BPE), the approach achieves reliable judgment quality while accepting significantly more evaluations than existing methods.
AINeutralarXiv – CS AI · 6d ago6/10
🧠Researchers introduce CROP, a statistical certification method for language model reasoning traces that identifies the longest reliable prefix before errors occur. The technique enables safer deployment of AI systems by providing rigorous guarantees about which intermediate reasoning steps can be trusted, while routing uncertain portions for human review or automated repair.
AINeutralarXiv – CS AI · May 276/10
🧠Researchers introduce MiRD, a two-stage framework that improves reliable prediction for open-ended question answering by separately addressing sampling failures and selection errors. The approach maintains calibration-set integrity while controlling hallucinations in AI models, outperforming existing conformal prediction methods across multiple datasets and models.
AIBullisharXiv – CS AI · May 276/10
🧠Researchers propose LEC (Linear Expectation Constraints), a framework for controlling prediction errors in foundation models by setting user-specified risk thresholds. The method enables selective prediction systems and multi-model routing architectures to maintain statistical guarantees on error rates while maximizing the number of accepted predictions, with applications spanning QA and vision tasks.
AIBullisharXiv – CS AI · May 96/10
🧠Researchers propose a two-stage approach to improve reliability in retrieval-augmented generation (RAG) systems by using conformal prediction to filter retrieved content and an attention-based classifier to detect factual inconsistencies. The framework achieves up to 6% answer quality improvement and 77% inconsistency detection, advancing toward certified RAG systems for production AI applications.
AINeutralarXiv – CS AI · May 76/10
🧠Researchers propose Adaptive Conformal Semantic Entropy (ACSE), a novel method for quantifying uncertainty in large language model outputs by measuring semantic diversity rather than relying solely on lexical or probabilistic measures. The approach uses conformal calibration to provide statistical guarantees on error rates, demonstrating significant performance improvements over existing uncertainty quantification baselines.
AINeutralarXiv – CS AI · Apr 206/10
🧠Researchers propose a conformal prediction framework for large language models that uses internal neural representations rather than surface-level outputs to assess reliability and uncertainty. The Layer-Wise Information scoring method improves prediction validity under distribution shift while maintaining competitive performance, addressing a critical challenge in deploying LLMs where traditional uncertainty signals become unreliable.
AINeutralarXiv – CS AI · Mar 55/10
🧠Researchers introduce zono-conformal prediction, a new uncertainty quantification method for machine learning that uses zonotope-based prediction sets instead of traditional intervals. The approach is more computationally efficient and less conservative than existing conformal prediction methods while maintaining statistical coverage guarantees for both regression and classification tasks.
AIBullisharXiv – CS AI · Mar 37/107
🧠Researchers have developed a conformal policy control method that enables AI agents to safely explore new behaviors while maintaining strict safety constraints. The approach uses safe reference policies as probabilistic regulators to determine how aggressively new policies can act, providing finite-sample guarantees without requiring specific model assumptions or hyperparameter tuning.
AINeutralarXiv – CS AI · Mar 35/104
🧠Researchers developed a conformal prediction framework for Large Language Models used in medical entity extraction, testing on FDA drug labels and radiology reports. The study found that model calibration varies significantly across clinical domains, with models being underconfident on structured data but overconfident on free-text reports, achieving 90% target coverage with 9-13% rejection rates.
