A Comparative Study of Student Perspectives on Technical Writing Feedback Quality: Evaluating LLMs, SLMs, and Humans in Computer Science Topics

A research study compares feedback quality from locally-hosted small language models (SLMs), commercial LLMs like GPT-4, and human instructors across computer science courses. The findings show that quantized Llama-3.1 matched commercial LLM performance while offering privacy and cost advantages, though human feedback remained superior for specialized writing tasks.

This academic research addresses a genuine tension in educational technology: how to scale personalized feedback without incurring prohibitive costs or privacy risks associated with commercial AI services. The study's deployment of a quantized Llama-3.1 model across three different course contexts—introductory programming, operating systems, and a writing seminar—provides empirical grounding for what has largely been theoretical discussion about open-source model viability in educational settings.

The broader context reflects growing institutional hesitance toward vendor lock-in with proprietary LLM providers. Universities and educational organizations face increasing scrutiny over data privacy, budget constraints, and student data governance. Local SLM deployment represents a pragmatic middle ground, enabling institutions to leverage AI capabilities while maintaining data sovereignty and reducing API costs to near-zero marginal expense.

For the EdTech and educational software market, this validates a tiered feedback approach where different tasks benefit from different AI tiers. The research suggests that foundational structural feedback—the most time-consuming and scalable component—can be effectively handled by efficient local models, freeing human instructors to focus on conceptual depth and specialized guidance. This has immediate implications for educational institutions evaluating their AI infrastructure investments and for companies building feedback systems.

The study's quantitative findings on readability and actionability metrics suggest open-source models have narrowed the capability gap with commercial alternatives in specific domains. Future implementations will likely test this framework across different disciplines and institutional settings, potentially establishing open-source SLMs as standard infrastructure for scalable feedback systems in academic environments.

• →Quantized Llama-3.1 matched GPT-4 performance while eliminating privacy concerns and API costs for educational institutions.
• →Students rated local SLM feedback higher for readability and actionability in technical courses compared to commercial alternatives.
• →A tiered pedagogical framework—AI for structural feedback, humans for conceptual guidance—optimizes resource allocation and educational outcomes.
• →Local SLM deployment represents a viable alternative to commercial LLM APIs for foundational feedback at scale.
• →Human instructors remain essential for specialized, discipline-specific writing tasks despite advances in AI feedback quality.