From Learning Resources to Competencies: LLM-Based Tagging with Evidence and Graph Constraints

Researchers developed an LLM-based pipeline that automatically tags learning resources with competencies from structured frameworks, combining language models with graph constraints and evidence extraction. The system achieved strong performance metrics (0.57 micro-F1, 0.82 MRR) while providing transparent, auditable evidence spans—outperforming traditional baselines and addressing the labor-intensive challenge of manual resource tagging in educational systems.

This research addresses a persistent friction point in educational technology: the gap between rich learning content and structured competency frameworks that enable intelligent search and curriculum design. Manual tagging at scale is prohibitively expensive, while fully automated approaches typically operate as black boxes, making institutional adoption difficult. The proposed solution leverages LLMs strategically as constrained taggers rather than sole decision-makers, a methodologically sound approach that balances automation with interpretability.

The pipeline's design reflects maturity in practical AI deployment. By segmenting resources into pedagogical fragments and retrieving candidate competencies using BM25 retrieval enriched with graph context, the system reduces the LLM's decision space and computational cost. The inclusion of evidence span extraction—where the model highlights supporting text—directly addresses institutional skepticism about AI recommendations in educational contexts, enabling human auditors to validate and learn from decisions.

For the educational technology sector, this work signals viable pathways for competency-based learning at scale. The 0.57 micro-F1 performance, while not perfect, substantially exceeds zero-shot and few-shot LLM baselines and supervised classifiers, suggesting the hybrid approach genuinely adds value. The MRR of 0.82 indicates strong ranking capability, critical for search applications. Institutions using Learning Management Systems can now feasibly migrate toward competency-centric architectures that were previously inaccessible due to tagging bottlenecks.

The research opens opportunities for EdTech vendors to implement competency alignment as a core feature, potentially enabling new curriculum analytics, personalized learning pathways, and employer-aligned skill tracking. Larger question remains whether evidence spans sufficiently satisfy institutional governance requirements around AI decision-making in educational contexts.

• →LLM-based tagging with graph constraints and evidence extraction achieves 0.57 micro-F1, outperforming baselines while maintaining human-auditable transparency.
• →The hybrid BM25+Graph+LLM approach reduces computational cost and decision space by pre-filtering candidate competencies before LLM evaluation.
• →Evidence span extraction enables institutional trust in automated competency tagging by providing mechanically traceable justifications for assignments.
• →Resource-level aggregation and graph-based refinement improve practical usability for competency-based search and curriculum analytics in educational systems.
• →The methodology demonstrates viable pathways for scaling competency frameworks in Learning Management Systems without prohibitive manual labor costs.