#black-box-methods News & Analysis

3 articles tagged with #black-box-methods. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AINeutralarXiv – CS AI · Jun 197/10

🧠

A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Models

Researchers present a comprehensive evaluation framework for black-box uncertainty estimation methods in large language models, benchmarking 24 methods across 4 models and datasets. The study reveals that no single approach dominates universally, but hybrid methods combining multiple uncertainty signals and candidate-reasoning approaches consistently outperform others, addressing critical gaps in trustworthy LLM deployment.

AINeutralarXiv – CS AI · Jun 96/10

🧠

BEACON: Behavioral Entropy Aggregation for Cross-Model Hallucination Detection in Large Language Models

Researchers introduce BEACON, a black-box hallucination detection framework for large language models that achieves 81.23% accuracy by analyzing model outputs without requiring internal access. The method combines multiple uncertainty signals including semantic entropy and consistency checks, outperforming existing baselines and offering practical deployment options across commercial LLM APIs.

AINeutralarXiv – CS AI · May 286/10

🧠

Entropy Distribution as a Fingerprint for Hallucinations in Generative Models

Researchers propose Calibrated Entropy Score (CES), a novel method for detecting hallucinations in large language models using entropy distribution patterns from a single forward pass. The technique achieves performance comparable to computationally expensive multi-sample methods while requiring only black-box access to token logits, with formal mathematical guarantees for detection accuracy.

🏢 Perplexity