General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks

Researchers introduce General365, a benchmark revealing that leading LLMs achieve only 62.8% accuracy on general reasoning tasks despite excelling in domain-specific domains. The findings highlight a critical gap: current AI models rely heavily on specialized knowledge rather than developing robust, transferable reasoning capabilities applicable to real-world scenarios.

The General365 benchmark addresses a fundamental blind spot in LLM evaluation. While models like GPT-4 and Claude demonstrate near-perfect performance on math and physics problems, this new research exposes their struggle with broader reasoning tasks that require only K-12 level knowledge. This discrepancy suggests that impressive benchmark scores in specialized domains may mask underlying weaknesses in generalized reasoning ability. The benchmark's design—1,095 problems across eight categories with constraints, nested logic, and semantic interference—mirrors real-world complexity that current models fail to handle consistently.

This research emerges amid growing scrutiny of how LLMs are evaluated. Industry leaders have faced criticism for cherry-picking benchmarks, and General365 provides a more holistic assessment tool. The stark performance gap (62.8% vs. near-perfect scores elsewhere) validates concerns that model capabilities don't translate seamlessly across different problem domains. For developers and organizations deploying LLMs in production, this signals caution: models optimized for specific tasks may fail catastrophically when reasoning requirements shift.

The market implications extend beyond academic interest. Companies investing billions in LLM infrastructure now face evidence that current architectures have fundamental reasoning limitations. This justifies continued R&D spending to develop more generalizable models, potentially benefiting research-focused AI companies and those pursuing novel training methodologies. The benchmark itself becomes valuable intellectual property for evaluating next-generation systems.

Looking ahead, General365 establishes a standard for measuring genuine reasoning progress. Success on this benchmark will differentiate leaders from followers in the AI race, making it essential for startups and labs claiming reasoning breakthroughs.

• →Top LLMs achieve only 62.8% accuracy on General365 despite near-perfect performance on specialized math and physics benchmarks.
• →Current LLMs demonstrate domain-dependent reasoning abilities rather than robust general-purpose reasoning applicable to real-world scenarios.
• →The benchmark decouples reasoning assessment from specialized knowledge by restricting problems to K-12 level content, revealing true reasoning limitations.
• →General365's 1,095 problems with complex constraints and semantic interference mirror real-world complexity that exposes model weaknesses.
• →The research establishes a new evaluation standard that will differentiate AI models and justify continued investment in more generalizable reasoning systems.