Can Large Language Models Derive New Knowledge? A Dynamic Benchmark for Biological Knowledge Discovery

Researchers have developed DBench-Bio, a dynamic benchmark system that automatically evaluates AI's ability to discover new biological knowledge using a three-stage pipeline of data acquisition, question-answer extraction, and quality filtering. The benchmark addresses the critical problem of data contamination in static datasets and provides monthly updates across 12 biomedical domains, revealing current limitations in state-of-the-art AI models' knowledge discovery capabilities.

• →DBench-Bio is the first dynamic, automated benchmark specifically designed to evaluate AI's biological knowledge discovery capabilities.
• →The system addresses data contamination issues in static benchmarks by using a continuously updated pipeline from authoritative paper abstracts.
• →Current state-of-the-art AI models show significant limitations in discovering genuinely new knowledge according to benchmark evaluations.
• →The benchmark covers 12 biomedical sub-domains and provides monthly updates to ensure relevance and currency.
• →This framework establishes a new standard for assessing AI knowledge discovery capabilities in scientific research.