Frontier LLM-based agents can overcome the ontology curation bottleneck for natural phenotypes

Frontier large language models from Anthropic and OpenAI have demonstrated competitive performance with human experts at annotating natural phenotypes to ontology terms, a previously labor-intensive bottleneck in biological research. When evaluated against the same Gold Standard benchmark used in a 2018 study, these AI agents performed within the range of trained human curators and substantially outperformed prior NLP tools, suggesting significant potential to scale phenotype annotation workflows.

The phenotype annotation challenge represents a critical scaling problem in comparative biology. Researchers need to link free-text descriptions of organism characteristics to standardized ontology terms—a process requiring deep domain expertise and manual curation. The 2018 Dahdul benchmark established that machine consistency lagged significantly behind human-human agreement, making automation seemingly unviable for this task.

This research reveals a dramatic shift in AI capability since that benchmark. Five frontier LLMs operating as autonomous agents—equipped with relevant ontologies (UBERON, PATO, BSPO, GO), annotation guidelines, and publication context—matched the performance range of human experts. The top-performing agents approached the best human curator's accuracy, while all agents decisively surpassed Semantic CharaParser, the previous state-of-art NLP tool.

The breakthrough stems from modern LLMs' superior language understanding, reasoning capabilities, and ability to follow complex instructions within structured workflows. By providing agents with the same resources human curators access—source PDFs, standardized guides, validation scripts—the research demonstrates that performance gaps narrow considerably when context is rich and task structure is clear.

For the broader scientific infrastructure, this finding has immediate implications. Phenotype annotation bottlenecks have constrained morphological data integration across studies, limiting comparative evolutionary and developmental research. Automating this process could accelerate knowledge synthesis at scale. However, the inability of agents to exceed the best human performers suggests hybrid workflows may prove optimal—using AI for initial annotation with expert review for validation. This positions AI as a force multiplier for expert curators rather than a replacement, potentially reducing time-to-completion while maintaining quality standards.

• →Frontier LLMs now perform competitively with trained human biocurators on phenotype annotation tasks, matching inter-curator variability ranges.
• →AI agents substantially outperformed prior NLP tools (Semantic CharaParser) across all evaluated metrics when given proper context and resources.
• →The breakthrough demonstrates that modern LLMs can handle complex domain-specific annotation when equipped with relevant ontologies, guidelines, and publication data.
• →This removes a major scaling bottleneck in comparative morphology research, enabling faster cross-study data integration.
• →Hybrid human-AI workflows likely represent the optimal approach, using agents for initial annotation and experts for validation rather than full automation.