The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing

Researchers present evidence that safe autonomous AI prescribing requires three architectural safeguards: calibrated confidence thresholds, differentiated uncertainty communication, and decision transparency. A clinician survey of 136 U.S. prescribers reveals these features would substantially increase adoption but would effectively reduce AI systems from true autonomous agents to supervised decision-support tools.

The transition of AI systems from advisory to autonomous roles in medication prescribing represents a critical inflection point for healthcare AI regulation. Recent legislative initiatives like H.R. 238 and Utah's prescription-renewal pilot authorize AI to operate as independent prescribing agents, yet existing regulatory frameworks rely solely on aggregate performance metrics without addressing the operational mechanisms that clinicians require for safe delegation. This research bridges that gap by identifying three non-negotiable architectural requirements grounded in both technical rigor and clinical practice realities.

The distinction between epistemic uncertainty (model ignorance) and aleatoric uncertainty (genuine clinical ambiguity) proves operationally decisive. Clinicians in the survey demonstrated sophisticated reasoning: they preferred summary views of competing treatment options when uncertainty reflected genuine clinical complexity, but shifted toward complete abstention when gaps in model knowledge created risk. This preference reveals a crucial insight—clinicians do not resist AI autonomy uniformly but calibrate trust based on uncertainty type and explainability.

The liability allocation dimension reshapes how autonomy should be structured. Clinicians proved willing to accept additional responsibility only when system transparency enabled substantive judgment under acknowledged uncertainty. This finding directly challenges conventional AI autonomy concepts, suggesting that regulatory approval of truly autonomous prescribing systems may face sustained clinical resistance regardless of performance metrics. The requirement for inferential transparency at decision moments effectively recreates human oversight at critical junctures.

These findings carry implications for healthcare AI deployment velocity. Systems meeting the identified requirements function as heavily supervised tools rather than autonomous agents, potentially limiting the efficiency gains that proponents of autonomous prescribing envision. However, this architectural constraint may accelerate genuine adoption by addressing clinician concerns systematically rather than through regulatory mandate alone.

• →Clinicians require calibrated per-prediction confidence thresholds and escalation mechanisms before accepting autonomous AI prescribing systems.
• →Differentiation between epistemic uncertainty (model ignorance) and aleatoric uncertainty (clinical ambiguity) fundamentally shapes clinician decision-making and system design requirements.
• →Liability acceptance by clinicians depends directly on inferential transparency enabling substantive judgment rather than blind reliance on AI recommendations.
• →Architectural requirements for safe autonomous prescribing effectively reduce true autonomy, converting systems into heavily supervised decision-support tools.
• →Regulatory frameworks must address operational safeguards beyond aggregate performance metrics to achieve clinician adoption and safe deployment.