Compass: Navigating Global Marine Lead Data Integration through Expert-Guided LLM Agent

Researchers introduced Compass, an LLM agent framework that extracts marine lead data from 230,000+ academic papers without fine-tuning, successfully creating the largest integrated marine lead database with 3,751 previously uncatalogued records and 92% accuracy. The expert-guided approach demonstrates how domain-specific knowledge can overcome LLM hallucinations in high-stakes scientific applications.

Compass addresses a fundamental challenge in scientific research: converting unstructured academic literature into actionable datasets. Marine lead isotopes serve as critical environmental tracers for understanding ocean circulation patterns and tracking anthropogenic pollution, yet relevant observational data remains fragmented across thousands of papers. Traditional manual extraction is prohibitively expensive and time-consuming, while general-purpose LLMs lack the domain expertise to extract scientifically valid information reliably.

The framework's innovation lies in its expert-guided adaptation methodology. Rather than relying solely on LLM capabilities, researchers co-designed a Knowledge Tree with domain experts that decomposes complex extraction tasks into verifiable, step-by-step reasoning processes. This approach constrains the model's decision-making within scientifically valid parameters, dramatically reducing hallucinations that plague LLMs in specialized fields.

The deployment results are significant for the geosciences community. Successfully extracting 3,751 previously inaccessible records and achieving 92% expert-verified accuracy demonstrates scalable data discovery at unprecedented speed and accuracy. Notably, the integrated database now covers previously under-sampled regions including the East China Sea and Southern Ocean, addressing geographic biases in existing datasets.

Beyond marine science, Compass establishes a replicable pattern for bridging general-purpose AI and specialized scientific domains. The interactive visualization platform facilitates open access, enabling broader research participation. This work suggests that expert-guided LLM agents could accelerate data integration across other data-rich but fragmented scientific domains, from climate research to materials science, potentially unlocking billions in research value currently trapped in unstructured literature.

• →Compass successfully extracted 3,751 marine lead records from 230,000+ papers with 92% accuracy without model fine-tuning
• →Expert-guided Knowledge Trees enable LLMs to maintain scientific validity while avoiding hallucinations in specialized domains
• →The largest integrated marine lead database now covers previously under-sampled ocean regions like the East China Sea and Southern Ocean
• →The framework demonstrates a scalable pattern for applying general-purpose LLMs to high-stakes scientific data extraction across multiple disciplines
• →Open-source visualization platform released to facilitate broader scientific access and collaborative research