Neither Replacement nor Panacea: Comparing LLM-Based Conversational and Graphical Decision Support in Industrial Tasks

A study comparing LLM-based conversational interfaces with traditional dashboards for industrial decision-making found that conversational AI reduces perceived mental workload and speeds up simple tasks, but provides no consistent advantage in decision accuracy and loses effectiveness as task complexity increases. The research suggests conversational agents complement rather than replace visual dashboards for manufacturing decision support.

This research addresses a critical question in enterprise AI adoption: whether conversational interfaces powered by large language models can meaningfully replace traditional data visualization tools in high-stakes operational environments. The study's 2x3 factorial design with 134 industrial decision-makers across varying task complexities provides empirical evidence that challenges both the hype surrounding conversational AI and dismissals of its utility.

The findings emerge from a broader industry trend of AI vendors positioning language models as universal solutions for data access and interpretation. Manufacturing firms face mounting pressure to extract actionable insights from increasingly complex datasets, creating strong incentives to adopt easier-to-use interfaces. However, this research demonstrates that interface effectiveness is task-dependent rather than universally superior.

For enterprise software vendors and manufacturers, the implications are substantial. Organizations cannot simply replace dashboards with chatbots and expect equivalent decision quality. The research indicates that conversational agents excel at reducing cognitive friction for straightforward queries but falter when decisions require sustained inspection of multiple data relationships—precisely where visual persistence matters. Mental workload reduction in simple tasks carries limited business value if accuracy deteriorates on complex decisions.

The finding that data literacy did not moderate interface effects challenges a common assumption among product teams: that better-trained users would extract maximum value from conversational AI. Looking forward, the most promising path forward appears to be hybrid systems combining conversational access for initial exploration with visual dashboards for complex analysis. Organizations implementing LLM-based decision support should architect solutions that intelligently route users between modalities rather than treating conversational interfaces as wholesale replacements for existing visual systems.

• →Conversational AI reduces perceived mental workload for simple tasks but provides no advantage as decision complexity increases
• →LLM-based interfaces achieved faster task completion only on less demanding scenarios, with benefits diminishing significantly for complex decisions
• →Neither conversational nor graphical interfaces produced consistent superiority in decision accuracy across all task types
• →Data literacy levels did not meaningfully moderate the effectiveness differences between interface types
• →Hybrid systems combining conversational and visual interfaces appear more promising than replacing dashboards entirely with chatbots