🧠 AI⚪ Neutral

Engineering Reasoning and Instruction (ERI) Benchmark: A Large Taxonomy-driven Dataset for Foundation Models and Agents

arXiv – CS AI|MZ Naser, Ahmad Bani Awwad, Zoie McCreery, Radwa Eissa, Ahmad Naser, Gianluca Cusatis, Andrew Metcalf, Kapil Madathil, Jamal Abdalla, Venkatesh Kodur, Mohammad Reza Saeb|March 4, 2026 at 05:00 AM|1 views

🤖AI Summary

Researchers released the ERI benchmark, a comprehensive dataset spanning 9 engineering fields and 55 subdomains to evaluate large language models' engineering capabilities. The benchmark tested 7 LLMs across 57,750 records, revealing a clear three-tier performance structure with frontier models like GPT-5 and Claude Sonnet 4 significantly outperforming mid-tier and smaller models.

Key Takeaways

→ERI benchmark covers 9 engineering fields with 57,750 records across undergraduate, graduate, and professional difficulty levels.
→Frontier models (GPT-5, Claude Sonnet 4, DeepSeek V3.1) achieved mean scores above 4.30 on a five-point scale.
→Mid-tier and smaller models showed progressively higher failure rates on graduate-level engineering questions.
→Researchers developed a validation protocol that bounds hallucination risk to 1.7% through cross-provider independence.
→The dataset is publicly released with evaluation tools to enable reproducible AI model testing in engineering applications.

#llm-benchmark #engineering-ai #model-evaluation #academic-research #ai-performance #dataset-release #frontier-models #validation-protocol

Read Original →via arXiv – CS AI

Act on this with AI

Stay ahead of the market.

Connect your wallet to an AI agent. It reads balances, proposes swaps and bridges across 15 chains — you keep full control of your keys.

Connect Wallet to AI →How it works

AI55m ago

Warren Buffett complained for decades that boosting profits by excluding exec stock comp was ‘cynical’—Nvidia just surprised Wall Street and agreed

Nvidia surprised Wall Street by agreeing to include executive stock compensation in its profit calculations, addressing a decades-old complaint by Warren Buffett about excluding such costs. This accounting change will likely boost Nvidia's credibility with investors while potentially pressuring competitors to follow suit.

AI3h ago

A Neuropsychologically Grounded Evaluation of LLM Cognitive Abilities

Researchers introduced NeuroCognition, a new benchmark for evaluating LLMs based on neuropsychological tests, revealing that while models show unified capability across tasks, they struggle with foundational cognitive abilities. The study found LLMs perform well on text but degrade with images and complexity, suggesting current models lack core adaptive cognition compared to human intelligence.

AI3h ago

Estimating Visual Attribute Effects in Advertising from Observational Data: A Deepfake-Informed Double Machine Learning Approach

Researchers developed DICE-DML, a new framework that uses deepfake technology and machine learning to measure causal effects of visual attributes in digital advertising. The method addresses bias issues in standard approaches when analyzing how image elements like skin tone affect consumer engagement on social media platforms.