Auditing medical multi-agent AI reveals risks of false consensus

Researchers introduced MedAgentAudit, a framework that reveals critical safety failures in medical multi-agent AI systems, finding that collaborative AI architectures frequently exhibit unsupported observations, evidence avoidance, and decision-making biases rather than genuine reasoning. The study across 14,400 cases and six AI architectures demonstrates that consensus-based medical AI systems are unreliable for clinical use without fundamental process-level improvements.

The research exposes a fundamental disconnect between how medical AI systems perform and how they actually function collaboratively. While developers prioritize accuracy metrics in final outputs, clinicians operating these systems need assurance that agents properly evaluated evidence, managed disagreement, and maintained transparency about uncertainty. MedAgentAudit's audit framework identifies ten recurring failure modes, including unsupported observations affecting 16.63% of cases, evidence re-examination occurring in only 1.58% of discussions, and authority bias increasing from 35.30% to 68.75% across decision rounds.

This work reflects growing concerns about deploying large language models in high-stakes medical settings. Multi-agent architectures were theoretically designed to emulate multidisciplinary clinical teams through specialist roles and peer review. However, the study reveals agents primarily repeat initial positions rather than genuinely collaborating, suggesting current LLMs lack the epistemic rigor necessary for medical decision support. The problem intensifies during synthesis phases, where systems substitute majority voting or authority deference for actual evidence evaluation.

For the medical AI industry and healthcare institutions, these findings represent a critical inflection point. Organizations investing in multi-agent clinical systems must now account for process-level auditing alongside accuracy metrics, substantially increasing implementation complexity and cost. The research establishes that transparency and auditability—not just prediction accuracy—define safe agentic systems in medicine. Healthcare regulators and institutional review boards will likely demand similar audit frameworks before approving AI systems for clinical deployment, fundamentally reshaping how medical AI evaluation occurs.

• →Multi-agent medical AI systems exhibit frequent collaborative failures including unsupported observations, evidence avoidance, and bias that standard accuracy metrics fail to detect.
• →Authority bias in medical AI synthesis increases dramatically across decision rounds (35.30% to 68.75%), substituting consensus for evidence-based reasoning.
• →Agents repeat initial viewpoints in 98.42% of discussions rather than re-examining evidence or activating specialist reasoning, undermining multidisciplinary team designs.
• →Process-level auditing through frameworks like MedAgentAudit is essential for clinical deployment, shifting evaluation from output scoring to safety and accountability.
• →Current multi-agent architectures lack sufficient epistemic rigor for high-stakes medical decision support without fundamental improvements to reasoning and transparency.