#user-study News & Analysis

8 articles tagged with #user-study. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

8 articles

AINeutralarXiv – CS AI · Mar 277/10

🧠

Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understanding

A user study with 200 participants found that while explanation correctness in AI systems affects human understanding, the relationship is not linear - performance drops significantly at 70% correctness but doesn't degrade further below that threshold. The research challenges assumptions that higher computational correctness metrics automatically translate to better human comprehension of AI decisions.

AIBullisharXiv – CS AI · Jun 256/10

🧠

AI Coaching for Accelerating Human Skill Development with Reinforcement Learning

Researchers present a reinforcement learning framework for AI coaching that balances skill acceleration with learner independence by strategically withdrawing assistance as competence develops. A user study on drone racing demonstrates the approach significantly outperforms existing AI coaching baselines, addressing the critical problem of skill atrophy caused by over-reliance on AI assistance.

AINeutralarXiv – CS AI · Jun 96/10

🧠

Seeing the Hivemind: A Consensus-Aware Interaction Technique for Mitigating AI Homogenization

Researchers introduced the Semantic Repulsion Technique (SRT) to combat AI homogenization in creative writing tasks, demonstrating that the method increases semantic diversity by 85-167% while reducing consensus phrases by 43-95%. A user study with 16 participants showed SRT outputs received higher usefulness and coherence ratings, with 68.8% willing to adopt it versus 18.8% for baseline systems, suggesting AI tools can enhance creativity without sacrificing readability.

AIBullisharXiv – CS AI · Jun 46/10

🧠

Characterizing initial human-AI proof formalization workflows

Researchers conducted mixed-methods studies on how mathematicians use AI tools to formalize proofs, finding that users prefer AI assistance while maintaining high-level control over proof discovery. A controlled user study showed participants achieved higher formalization accuracy with AI access than without, despite current tool limitations.

AINeutralarXiv – CS AI · May 285/10

🧠

From Instructor to Collaborator: What a 90-Participant Study Reveals about Human-Agent Collaboration in a Mobile Serious Game

A PhD study of 90 participants compared human-like spoken embodied conversational agents versus text-based agents in a mobile educational game about UK currency. Results showed statistically significant user preference for highly human-like agents, with implications for designing collaborative human-agent systems in educational contexts.

AINeutralarXiv – CS AI · Apr 105/10

🧠

Analyzing Multimodal Interaction Strategies for LLM-Assisted Manipulation of 3D Scenes

Researchers conducted an empirical user study examining how 12 participants interact with LLM-assisted 3D scene editing systems in immersive environments. The study combined quantitative usage data with qualitative feedback to identify interaction patterns, barriers, and design recommendations for future LLM-integrated 3D content creation tools.

AINeutralarXiv – CS AI · Mar 44/103

🧠

A Directed Graph Model and Experimental Framework for Design and Study of Time-Dependent Text Visualisation

Researchers developed a framework to study how people interpret time-dependent text visualizations using directed graph models and synthetic data generated by LLMs. The study found that users struggle to identify predefined patterns in text relationships, suggesting visualization tools may need personalized approaches rather than one-size-fits-all solutions.

AINeutralarXiv – CS AI · Feb 274/106

🧠

CoLyricist: Enhancing Lyric Writing with AI through Workflow-Aligned Support

CoLyricist is a new AI-assisted lyric writing tool designed around the typical workflows of experienced lyricists, featuring four key stages: Theme Setting, Ideation, Drafting Lyrics, and Melody Fitting. A user study with 16 participants showed the tool enhances songwriting across skill levels, with novices particularly appreciating the Melody-Fitting feature and experienced users valuing the Ideation support.