Mind the Gap: Can Frontier LLMs Pass a Standardized Office Proficiency Exam?

Researchers benchmarked 7 frontier LLMs against China's National Computer Rank Examination, a standardized office proficiency test with 200 practical tasks across Word, Excel, and PowerPoint. Single-turn models achieved only 36.6% accuracy, while advanced agentic systems with iterative feedback reached 68.8%, revealing significant gaps in LLM-based office automation despite recent code-generation improvements.

The study exposes a critical gap between theoretical LLM capabilities and practical real-world performance in professional environments. While large language models have demonstrated impressive code generation and reasoning abilities, their deployment for automating complex office workflows remains unreliable. The 68.8% score achieved by the best agentic system, though substantially higher than single-turn models, falls short of the 95.5% human reference baseline, indicating that current systems cannot be trusted for mission-critical document automation tasks.

This research responds to the accelerating deployment of LLM agents in enterprise settings, where organizations increasingly seek to automate productivity workflows. Office automation requires a constellation of capabilities—long-horizon planning, precise parameter configuration, and multi-application integration—that contemporary systems struggle to coordinate consistently. The NCRE evaluation framework provides a rigorous, standardized benchmark that transcends lab conditions, reflecting genuine professional requirements.

The findings carry significant implications for enterprise software vendors and AI application developers. Businesses considering LLM-based automation solutions should temper expectations about autonomous office task completion. The results suggest a maturity gap: while individual components function adequately, end-to-end workflows demand human oversight. This creates opportunities for hybrid systems combining human verification with AI assistance rather than full automation.

Future development should focus on improving iterative repair mechanisms, enhancing cross-application reasoning, and developing better error detection frameworks. As LLMs continue evolving, systematically tracking performance on standardized benchmarks like NCRE will be essential for assessing genuine progress in real-world automation capability.

• →Single-turn LLMs achieve only 36.6% on office automation tasks, exposing severe limitations in real-world deployment scenarios.
• →Advanced agentic systems with execution feedback reach 68.8%, but still fall significantly short of 95.5% human proficiency baseline.
• →Office automation requires coordinated long-horizon planning, parameter precision, and multi-application integration that current systems cannot reliably execute.
• →The NCRE benchmark provides a rigorous, standardized evaluation framework reflecting genuine professional requirements beyond typical lab conditions.
• →Enterprises should prioritize hybrid human-AI workflows over full automation until system reliability approaches operational thresholds.