A criterion for Artificial General Intelligence: hypothetic-deductive reasoning, tested on ChatGPT

Researchers propose hypothetic-deductive reasoning as a key criterion for Artificial General Intelligence, arguing that advanced AI systems must demonstrate causal reasoning and hypothesis testing across complex problem domains. Testing this framework on ChatGPT reveals the model has limited capacity for these reasoning types when problems increase in complexity, suggesting current large language models fall short of AGI-level reasoning capabilities.

This arXiv paper establishes a concrete benchmarking framework for evaluating whether AI systems have achieved reasoning capabilities essential for AGI. Rather than relying on vague metrics, the authors decompose advanced thinking into hypothetic-deductive reasoning—forming hypotheses about a problem, then deriving solutions—with causal reasoning as a fundamental proxy. This distinction matters because it moves the AGI discussion from philosophical territory into testable engineering problems.

The research builds on decades of cognitive science literature showing that humans solve novel problems through hypothesis generation and logical deduction. By formalizing this as a criterion, the paper provides researchers a reproducible testing methodology beyond benchmark scores. The ChatGPT analysis demonstrates that contemporary large language models struggle when problems require multi-step hypothesis formation or causal chain reasoning, particularly as complexity increases.

For the AI development community, this work highlights a genuine capability gap between current systems and true AGI. While ChatGPT excels at pattern matching and statistical inference from training data, it lacks the systematic hypothesis-testing approach humans employ for genuinely novel problems. This suggests that scaling parameters alone won't achieve AGI; architectural or training innovations specifically targeting causal and hypothetic-deductive reasoning are necessary.

The implications extend beyond academia. If this criterion gains acceptance, it could redirect AI safety research toward understanding how to build systems with robust causal reasoning, potentially uncovering failure modes in current approaches. Developers working on reasoning-intensive applications should expect that current models will require significant human oversight for complex problem-solving domains.

• →Hypothetic-deductive reasoning and causal reasoning are proposed as testable criteria for AGI rather than relying on subjective assessments.
• →ChatGPT demonstrates limited capacity for both reasoning types when problem complexity increases, suggesting current LLMs are far from AGI.
• →The framework provides a reproducible benchmark methodology that could standardize AGI evaluation across the research community.
• →Achieving AGI-level reasoning likely requires architectural innovations beyond parameter scaling in large language models.
• →This work suggests safety-critical AI applications will continue requiring human oversight for complex reasoning tasks.