AIBearisharXiv – CS AI · Apr 76/10
🧠Research reveals AI-generated economics papers significantly underperform human-authored publications, with idea quality representing the primary bottleneck (71% of the gap) rather than execution quality. Analysis of 953 papers shows human research achieves 47.1% exceptional probability versus 16.5% for AI, with only 0.8% of AI papers surpassing median human quality on both dimensions.
🧠 Gemini
AIBearisharXiv – CS AI · Apr 76/10
🧠Research reveals that Large Language Models (LLMs) experience greater performance degradation when facing English as a Second Language (ESL) inputs combined with typographical errors, compared to either factor alone. The study tested eight ESL variants with three levels of typos, finding that evaluations on clean English may overestimate real-world model performance.
AIBearisharXiv – CS AI · Apr 76/10
🧠New research reveals that Large Language Models (LLMs) exhibit cultural bias and Western defaultism when generating metaphors across different cultural contexts. The study found that LLMs act more as cultural translators using dominant Western frameworks rather than true culturally-aware reasoning systems, even when prompted with specific cultural identities.
AINeutralarXiv – CS AI · Apr 66/10
🧠Researchers introduce XpertBench, a new benchmark for evaluating Large Language Models on expert-level professional tasks across domains like finance, healthcare, and legal services. Even top-performing LLMs achieve only ~66% success rates, revealing a significant 'expert-gap' in current AI systems' ability to handle complex professional work.
AIBearisharXiv – CS AI · Apr 66/10
🧠Researchers introduce DeltaLogic, a new benchmark that tests AI models' ability to revise their logical conclusions when presented with minimal changes to premises. The study reveals that language models like Qwen and Phi-4 struggle with belief revision even when they perform well on initial reasoning tasks, showing concerning inertia patterns where models fail to update conclusions when evidence changes.
AIBearisharXiv – CS AI · Apr 66/10
🧠Research study reveals that Large Language Models can reproduce behavioral patterns but fail to accurately predict intervention effects. The study tested three LLMs on climate psychology interventions across 59,508 participants from 62 countries, finding that descriptive accuracy doesn't translate to causal prediction accuracy.
AIBearisharXiv – CS AI · Apr 66/10
🧠Researchers introduced ChomskyBench, a new benchmark for evaluating large language models' formal reasoning capabilities using the Chomsky Hierarchy framework. The study reveals that while larger models show improvements, current LLMs face severe efficiency barriers and are significantly less efficient than traditional algorithmic programs for formal reasoning tasks.
AIBearisharXiv – CS AI · Apr 66/10
🧠A new study reveals that large language models, despite excelling at benchmark math problems, struggle significantly with contextual mathematical reasoning where problems are embedded in real-world scenarios. The research shows performance drops of 13-34 points for open-source models and 13-20 points for proprietary models when abstract math problems are presented in contextual settings.
AIBearishTechCrunch – AI · Apr 56/10
🧠Microsoft's terms of service classify Copilot as being 'for entertainment purposes only,' indicating that even AI companies themselves warn users against blindly trusting AI model outputs. This aligns with broader industry cautions about AI reliability and the need for human oversight when using AI tools.
🏢 Microsoft
AIBearisharXiv – CS AI · Mar 276/10
🧠Research reveals that large language models (LLMs) struggle to maintain consistent internal beliefs or goals across multi-turn conversations, failing to preserve implicit consistency when not explicitly provided context. This limitation poses significant challenges for developing persona-driven AI systems that require stable personality traits and behavioral patterns.
AIBearisharXiv – CS AI · Mar 266/10
🧠Research reveals that multimodal language models have significant deficits in visuospatial perspective-taking, particularly in Level 2 VPT which requires adopting another person's viewpoint. The study used two human psychology tasks to evaluate MLMs' ability to understand and reason from alternative spatial perspectives.
AIBearisharXiv – CS AI · Mar 176/10
🧠Researchers introduced BrainBench, a new benchmark revealing significant gaps in commonsense reasoning among leading LLMs. Even the best model (Claude Opus 4.6) achieved only 80.3% accuracy on 100 brainteaser questions, while GPT-4o scored just 39.7%, exposing fundamental reasoning deficits across frontier AI models.
🧠 GPT-4🧠 Claude🧠 Opus
AIBearishThe Register – AI · Mar 176/10
🧠The article appears to discuss concerns about AI technology's current limitations and suggests that businesses may be overstating AI capabilities. A market correction or reassessment of AI's actual effectiveness may be approaching.
AIBearisharXiv – CS AI · Mar 126/10
🧠A research study analyzing 319 LLM-generated security patches found that only 24.8% achieve full correctness, with most failures due to semantic misunderstanding rather than syntax errors. LLMs preserve functionality well but struggle significantly with security fixes, with success rates varying dramatically by vulnerability type.
AIBearisharXiv – CS AI · Mar 96/10
🧠A comprehensive evaluation of Boltz-2, an AI-based drug discovery tool, reveals significant limitations in predicting protein-ligand binding structures and affinities. The study found only weak correlations with physics-based methods and concluded that while useful for initial screening, Boltz-2 lacks the precision required for reliable drug lead identification.
AIBearisharXiv – CS AI · Mar 96/10
🧠Researchers developed a new framework to assess moral competence in large language models, finding that current evaluations may overestimate AI moral reasoning capabilities. While LLMs outperformed humans on standard ethical scenarios, they performed significantly worse when required to identify morally relevant information from noisy data.
AINeutralarXiv – CS AI · Mar 96/10
🧠Researchers introduce KramaBench, a comprehensive benchmark testing AI systems' ability to execute end-to-end data processing pipelines on real-world data lakes. The study reveals significant limitations in current AI systems, with the best performing system achieving only 55% accuracy in full data-lake scenarios and leading LLMs implementing just 20% of individual data tasks correctly.
AIBearisharXiv – CS AI · Mar 36/104
🧠Researchers introduced SciTrek, a new benchmark for testing large language models' ability to perform numerical reasoning across long scientific documents. The benchmark reveals significant challenges for current LLMs, with the best model achieving only 46.5% accuracy at 128K tokens, and performance declining as context length increases.
$COMP
AIBearisharXiv – CS AI · Mar 36/104
🧠Researchers introduced HardcoreLogic, a benchmark of over 5,000 logic puzzles across 10 games to test Large Reasoning Models (LRMs) on non-standard puzzle variants. The study reveals significant performance drops in current LRMs when faced with complex or uncommon puzzle variations, indicating heavy reliance on memorized patterns rather than genuine logical reasoning.
AIBearisharXiv – CS AI · Mar 36/104
🧠Researchers introduced SimpleToM, a benchmark revealing that state-of-the-art language models can infer mental states but struggle to apply that knowledge for behavior prediction and judgment. The study exposes a critical gap between explicit Theory of Mind inference and implicit application in real-world scenarios.
AIBearisharXiv – CS AI · Mar 36/106
🧠Researchers compared human survey responses from 420 Silicon Valley developers with synthetic data from five leading LLMs including ChatGPT, Claude, and Gemini. While AI models produced technically plausible results, they failed to capture counterintuitive insights and only replicated conventional wisdom rather than revealing novel findings.
AINeutralarXiv – CS AI · Mar 36/108
🧠Researchers have identified a 'Paradox of Simplicity' in AI models where they excel at complex tasks but fail at simple ones like generating pure color images. A new benchmark called VIOLIN has been introduced to evaluate AI obedience and alignment with instructions across different complexity levels.
$RNDR
AIBearisharXiv – CS AI · Mar 37/108
🧠Research reveals that Large Language Models (LLMs) systematically fail at code review tasks, frequently misclassifying correct code as defective when matching implementations to natural language requirements. The study found that more detailed prompts actually increase misjudgment rates, raising concerns about LLM reliability in automated development workflows.
AIBearisharXiv – CS AI · Mar 36/106
🧠Researchers reveal that state-of-the-art Vision-Language-Action (VLA) models largely ignore language instructions despite achieving 95% success on standard benchmarks. The new LangGap benchmark exposes significant language understanding deficits, with targeted data augmentation only partially addressing the fundamental challenge of diverse instruction comprehension.
AIBearisharXiv – CS AI · Mar 36/106
🧠Research reveals that leading foundation models (LLMs) perform poorly on real-world educational tasks despite excelling on AI benchmarks. The study found that 50% of misalignment errors are shared across models due to common pretraining approaches, with model ensembles actually worsening performance on learning outcomes.
