ProjectionBench: Evaluating Scientific Hypothesis Generation in LLMs Under Progressive Information Disclosure

Researchers introduce ProjectionBench, a novel evaluation framework that tests large language models' scientific discovery capabilities by progressively revealing information about research problems. The benchmark assesses both innovative reasoning with minimal context and grounded hypothesis generation with full experimental details across 45 materials science papers, finding that GPT-5.4 and Gemini 3.1 Pro achieve strong alignment with ground-truth conclusions.

ProjectionBench addresses a critical gap in LLM evaluation by moving beyond knowledge recall benchmarks to assess genuine scientific reasoning and discovery potential. Traditional benchmarks primarily measure retrieval and multi-hop reasoning, but the field lacks systematic frameworks for evaluating the creative hypothesis generation essential to actual scientific work. This framework's progressive information disclosure methodology is particularly novel, measuring how models perform when forced to reason under uncertainty before receiving experimental details.

The research reflects growing recognition that LLMs may serve as scientific collaborators rather than mere information tools. As AI systems are increasingly proposed for autonomous research roles, rigorous evaluation frameworks become essential infrastructure. ProjectionBench's focus on semantic divergence from ground-truth conclusions and atomic claim evaluation provides quantifiable metrics for innovation and reasoning quality, moving beyond subjective assessment.

The findings suggest current frontier models possess meaningful scientific reasoning capabilities—particularly GPT-5.4's 0.7 F1 score alignment even with minimal context—but this remains preliminary across a limited domain sample of 45 papers. The results have implications for organizations developing AI research tools and academic institutions considering LLM integration into discovery workflows. A comprehensive evaluation across diverse scientific disciplines would strengthen confidence in these conclusions.

Future development should expand ProjectionBench across chemistry, physics, and biology domains while examining failure modes and systematic biases. Understanding where models diverge from expert reasoning will inform both LLM development and appropriate deployment boundaries for scientific applications.

• →ProjectionBench introduces progressive information disclosure methodology to evaluate both innovative and grounded scientific reasoning in LLMs.
• →GPT-5.4 demonstrates strongest performance, maintaining 0.7 F1 alignment with ground-truth conclusions even under minimal research context.
• →Current benchmarks lack frameworks for assessing genuine scientific discovery capabilities, making this work foundational for AI scientist development.
• →Semantic similarity evaluation of atomic claims enables objective comparison between model hypotheses and published research conclusions.
• →Limited evaluation across materials science suggests broader cross-discipline validation needed before confident deployment claims.