Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care

A study demonstrates that interactive dialogue between physicians and large language models significantly improves diagnostic accuracy in emergency medicine, with residents showing a 12.5% improvement on hard cases and standardized metrics confirming medium effect sizes across 52 clinical scenarios.

The research presents empirical validation of LLMs as practical clinical decision-support tools rather than theoretical possibilities. By requiring physicians to iteratively query an LLM with full clinical context while initially viewing only chief complaints, the MedSyn system mirrors real diagnostic workflows where information accumulates gradually. Residents improved hard-case accuracy from 58.9% to 73.4%, demonstrating particular value for less experienced clinicians who typically carry higher diagnostic uncertainty.

This work addresses a critical gap in medical AI adoption: most prior studies evaluate LLM performance on static benchmarks, but this research captures performance within actual physician workflows. The expertise-dependent dialogue patterns—seniors asking hypothesis-driven questions versus residents using broader queries—suggest the tool functions differently across experience levels, which has implications for implementation strategies in varied clinical settings.

The standardized any-match accuracy improvement of 15.6 percentage points and F1 gains for residents validate both the methodology and the practical utility. Cross-expertise concordance increasing by 14.5% indicates that LLM assistance narrows diagnostic gaps between junior and senior physicians, potentially reducing error variance in emergency departments. This convergence effect could have significant implications for patient outcomes in resource-constrained settings.

Clinical adoption remains dependent on addressing integration hurdles: real-time data accessibility, liability frameworks, and workflow disruption costs. The research suggests LLMs complement rather than replace physician judgment, requiring institutional investment in training physicians to interact effectively with AI. Future studies should examine longer-term adoption patterns and whether initial accuracy gains sustain after the novelty effect diminishes.

• →Residents improved hard-case diagnostic accuracy by 12.5 percentage points when using interactive LLM assistance with full clinical records.
• →Dialogue patterns reveal that senior physicians ask targeted, hypothesis-driven questions while residents use broader queries, suggesting expertise-dependent tool usage.
• →Standardized any-match accuracy improved 15.6%, with residents showing the largest F1-score gains of 13.8%, validating the tool's effectiveness across experience levels.
• →LLM assistance reduced diagnostic variability between junior and senior physicians, with cross-expertise concordance increasing 14.5%.
• →The study provides empirical evidence that interactive AI support enhances clinical reasoning in live workflows, moving beyond benchmark-only evaluation.