JobBench: Aligning Agent Work With Human Will

Researchers introduce JobBench, a new AI agent benchmark that evaluates 36 models across 130 tasks in 35 occupations based on what humans actually want delegated rather than pure economic value. The strongest model, Claude Opus, achieves only 45.9% accuracy, revealing significant gaps in current AI agent capabilities for real-world professional workflows.

JobBench represents a philosophical shift in how the AI community measures agent progress. Rather than optimizing for labor replacement and GDP impact, this benchmark prioritizes human-centered task delegation, asking what work humans genuinely want to offload versus what generates the most economic value. This distinction matters because it challenges the prevailing narrative that AI's primary purpose is productivity-driven automation. The benchmark's design reflects realistic professional environments, packaging 130 tasks with heterogeneous reference files and cluttered information streams—conditions that far exceed the controlled settings of prior benchmarks.

The research addresses a critical gap in current AI evaluation frameworks. Existing benchmarks often measure narrow capabilities in isolated domains, while JobBench spans diverse occupations and requires agents to navigate authentic workplace complexity. The fact-anchored rubric system with an average of 35.6 binary criteria per task ensures rigorous, defensible evaluation rather than subjective scoring. This methodological rigor has important implications for reliability and reproducibility.

The performance ceiling revealed by this benchmark—45.9% for the strongest model—signals that current AI agents struggle with practical professional delegation at scale. This gap provides realistic expectations for near-term AI deployment and highlights where substantial advancement is needed. For developers and enterprises planning AI integration, JobBench offers a more honest assessment of capabilities than previous benchmarks suggested.

Looking forward, this work may accelerate development of agents optimized for augmentation rather than replacement. If the research community adopts similar human-centered evaluation frameworks, it could reshape AI development priorities toward practical, workplace-specific improvements that genuinely enhance professional productivity rather than chase GDP metrics.

• →JobBench evaluates AI agents on 130 tasks across 35 occupations chosen for their delegation priority rather than economic value.
• →Claude Opus achieves only 45.9% accuracy on the benchmark, indicating significant capability gaps in real-world professional workflows.
• →The benchmark uses 35.6 binary criteria per task on average, ensuring rigorous evaluation grounded in factual rubrics.
• →The framework shifts evaluation focus from labor replacement to human-augmentation, challenging conventional AI development metrics.
• →Results suggest current AI agents remain unsuitable for widespread autonomous professional task delegation without substantial advancement.