AINeutralarXiv – CS AI · Mar 164/10
🧠
Human-Centered Evaluation of an LLM-Based Process Modeling Copilot: A Mixed-Methods Study with Domain Experts
Researchers conducted a mixed-methods study evaluating an LLM-powered BPMN modeling copilot with five domain experts, revealing acceptable usability (67.2/100) but significantly lower trust levels (48.8%). The study highlights critical reliability concerns and demonstrates the need for human-centered evaluation methods beyond automated benchmarking for LLM business tools.
🏢 Microsoft