From Instructor to Collaborator: What a 90-Participant Study Reveals about Human-Agent Collaboration in a Mobile Serious Game

A PhD study of 90 participants compared human-like spoken embodied conversational agents versus text-based agents in a mobile educational game about UK currency. Results showed statistically significant user preference for highly human-like agents, with implications for designing collaborative human-agent systems in educational contexts.

This empirical research addresses a fundamental question in human-computer interaction: how agent design features influence user engagement and task performance in collaborative scenarios. The study's within-subjects design with 90 participants provides robust statistical power, comparing two drastically different agent presentations—a sophisticated embodied conversational agent with speech capabilities against a minimal text-bubble interface—within a unified educational game context.

The research builds on decades of work in embodied conversational agents and human-AI collaboration, yet remains timely as organizations increasingly deploy AI agents for education, customer service, and knowledge work. The preference for human-like agents aligns with existing HCI literature suggesting anthropomorphic design increases engagement, but the study goes beyond simple preference metrics by examining how agent roles (instructor versus collaborator/shopkeeper) affect interaction patterns, dialogue management, and task completion.

For EdTech developers and AI product teams, these findings validate investments in conversational AI and embodied interfaces, suggesting that user experience improvements justify development complexity. The focus on mixed-initiative dialogue and repair mechanisms reveals practical challenges in real-world deployment—systems must gracefully handle user misunderstandings and task breakdowns without frustrating learners.

The research raises important questions about timing of agent interventions, managing user expectations around agent capabilities, and optimizing interactions based on assigned roles. Future work should explore whether human-likeness preferences hold across different cultural contexts, age groups, and task domains, and whether this preference translates to measurable learning outcomes rather than subjective satisfaction scores.

• →Highly human-like embodied agents showed statistically significant preference over text-based agents with large effect sizes in a 90-person study
• →Agent roles (instructor vs. collaborator) significantly influence how users interact with AI systems in goal-oriented educational tasks
• →Mixed-initiative dialogue and natural repair mechanisms become critical design considerations when agents share task responsibility with users
• →Anthropomorphic design trade-offs between development complexity and user engagement remain relevant for EdTech and enterprise AI applications
• →User expectations management and role-specific interaction patterns deserve investigation across different domains and demographics