Structuring versus Problematizing: How LLM-based Agents Scaffold Learning in Diagnostic Reasoning

Researchers developed PharmaSim Switch, an AI-powered educational platform that uses large language models to scaffold diagnostic reasoning in pharmacy technician training through two distinct pedagogical approaches: structuring and problematizing. A 63-student experiment found both methods effective, with structuring promoting more accurate participation and problematizing encouraging deeper constructive engagement, suggesting hybrid scaffolding strategies optimize learning outcomes.

This research addresses a critical gap in educational technology by empirically testing how different AI-driven pedagogical strategies influence diagnostic reasoning development. The study moves beyond simply deploying LLMs in education to examining the mechanisms through which conversational agents can scaffold learning, grounding the work in established learning science frameworks. The findings reveal nuanced differences in how scaffolding approaches shape cognitive engagement rather than crude performance metrics.

The broader context reflects growing recognition that AI tutoring systems must go beyond content delivery to facilitate deeper reasoning processes. Educational institutions increasingly seek evidence-based approaches for integrating LLMs, particularly in clinical and technical fields where diagnostic accuracy has real consequences. PharmaSim Switch demonstrates that conversational agents can be designed with pedagogical intent rather than generic responsiveness.

For edtech developers and AI vendors, these results provide actionable design principles: structuring approaches (which decompose problems systematically) enhance accurate task completion, while problematizing (which encourages questioning assumptions) drives constructive thinking. Educational institutions considering AI integration benefit from understanding that scenario complexity—not necessarily the sophistication of the AI agent—primarily determines learning outcomes. This suggests meaningful improvements don't require cutting-edge models but rather thoughtful pedagogical design.

Future work should explore how these scaffolding approaches transfer across domains beyond pharmacy, whether hybrid approaches combining both methods outperform either alone, and how individual differences in cognitive style interact with scaffolding type. The research suggests that AI's educational value emerges not from the technology itself but from deliberate alignment with learning science principles.

• →Both structuring and problematizing LLM scaffolding approaches effectively supported diagnostic reasoning, but through different cognitive engagement mechanisms.
• →Scenario complexity was the primary performance driver, outweighing prior knowledge and scaffolding approach selection.
• →Structuring promoted more accurate active participation while problematizing elicited deeper constructive engagement.
• →Hybrid pedagogical design combining multiple scaffolding approaches appears more effective than single-strategy implementations.
• →AI tutoring systems require deliberate alignment with learning science principles rather than generic conversational AI capabilities.