SKILLS: Structured Knowledge Injection for LLM-Driven Telecommunications Operations

Researchers introduced SKILLS, a benchmark framework testing whether large language models can execute telecommunications operations through APIs with or without structured domain guidance. The study evaluated 5 open-weight models across 37 telecom scenarios, showing consistent performance improvements when models were augmented with domain-specific guidance documents.

• →SKILLS framework tests LLM performance in telecommunications operations across 37 scenarios spanning 8 TM Forum Open API domains.
• →Models showed consistent performance improvements when provided with structured domain guidance versus baseline generic agents.
• →MiniMax M2.5 achieved the highest performance at 81.1% with skills, showing a 13.5 percentage point improvement.
• →The benchmark uses live mock API servers with production-representative data for realistic testing environments.
• →Results suggest LLMs require domain-specific guidance to reliably execute complex telecommunications workflows.