#black-box-models News & Analysis

3 articles tagged with #black-box-models. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AINeutralarXiv – CS AI · Jun 96/10

🧠

Diagnosing Multi-step Reasoning Failures in Black-box LLMs via Stepwise Confidence Attribution

Researchers introduce Stepwise Confidence Attribution (SCA), a framework for diagnosing where large language models fail in multi-step reasoning tasks without requiring access to the model's internal parameters. The method identifies problematic reasoning steps by measuring confidence alignment with consensus patterns across correct solutions, improving self-correction accuracy by up to 13.5%.

AINeutralarXiv – CS AI · Jun 56/10

🧠

Detecting Perspective Shifts in Multi-agent Systems

Researchers introduce Temporal Data Kernel Perspective Space (TDKPS), a framework for detecting behavioral changes in multi-agent AI systems across time. The method enables monitoring of black-box agent dynamics at both individual and group levels, addressing a critical gap in evaluating evolving generative agent systems.

AINeutralarXiv – CS AI · May 96/10

🧠

Measuring Black-Box Confidence via Reasoning Trajectories: Geometry, Coverage, and Verbalization

Researchers propose a novel black-box confidence estimation method for chain-of-thought reasoning that measures trajectory convergence rather than relying on expensive sampling. Testing across multiple benchmarks and AI models shows significant improvements over self-consistency baselines while requiring only 4 samples instead of 8, with potential applications for safer API-based AI deployment.

🧠 GPT-5🧠 Claude🧠 Sonnet