AI, Take the Wheel: What Drives Delegation and Trust in Human-Computer Cooperative Question Answering?

A research study examines how humans decide to trust and rely on AI systems in collaborative question-answering tasks, identifying two distinct reliance patterns: delegation (autonomous AI action) and adoption (evaluating AI suggestions). The findings reveal humans make suboptimal trust decisions, both under-utilizing correct AI suggestions and over-relying on misleading AI outputs, with confirmation bias playing a significant role in trust calibration failures.

This research addresses a critical gap in human-AI collaboration by studying trust decisions in realistic, competitive settings. Rather than treating reliance as monolithic, the authors distinguish between delegation—allowing AI to act without oversight—and adoption—consciously evaluating AI recommendations. This distinction matters because different trust failures require different interventions. The study's empirical foundation is robust: 23 expert humans competing across 24 matches with 16 AI agents generated 387 delegation decisions and 1,440 adoption decisions, providing statistically meaningful patterns.

The core finding reveals a paradox: humans perform better with AI assistance than alone, yet systematically make trust errors. They miss 3.9% of opportunities to accept correct AI suggestions while accepting 1.7% of misleading recommendations. Confirmation bias drives much of this dysfunction—when AI agrees with humans' initially incorrect answers, under-reliance jumps to 64.5%. This suggests humans treat AI agreement as validation rather than independent judgment.

For AI system developers and deployment teams, these results underscore that raw performance improvements alone cannot fix human-AI collaboration. The research points toward actionable interventions: calibrated confidence scores prevent overconfidence from driving over-reliance, evidence-grounded explanations help users understand AI reasoning beyond opaque suggestions, and trust-refinement mechanisms allow users to learn appropriate reliance patterns over time.

The findings carry implications beyond question-answering tasks. As AI systems proliferate across high-stakes domains—medical diagnosis, legal review, financial analysis—understanding and engineering better trust calibration becomes a fundamental safety requirement rather than a nice-to-have feature.

• →Humans demonstrate suboptimal AI trust, missing 3.9% of correct AI suggestions while over-relying on misleading outputs 1.7% of the time
• →Confirmation bias drives 64.5% under-reliance when AI agrees with humans' initial incorrect answers
• →AI system designers should implement calibrated confidence scores and evidence-grounded explanations to improve trust calibration
• →Human-AI teams outperform both humans and AI alone, but only when trust decisions are well-calibrated
• →Trust reliance involves two distinct mechanisms—delegation and adoption—requiring separate research attention and intervention strategies