AIBullisharXiv – CS AI · Jun 97/10
🧠A comprehensive empirical study analyzing 30 million US, UK, and Australian job postings finds that AI adoption increases demand for complementary human skills like analytical thinking and resilience rather than simply replacing workers. The research reveals significant wage premiums for these soft skills in AI-adjacent roles and spillover effects where AI diffusion reduces demand for substitutable tasks across entire industries and regions.
AIBearisharXiv – CS AI · Jun 236/10
🧠A rigorous empirical study challenges claims that large language models improve hyperparameter optimization for tabular data, finding that LLM advisors' apparent advantage comes entirely from a fixed default configuration seed, not the model itself. Classical search methods with the same seed match or outperform LLM approaches within a handful of evaluations, suggesting LLM-based HPO systems offer no meaningful generalization benefit.
AIBearisharXiv – CS AI · Jun 116/10
🧠Researchers empirically tested whether open-source LLM-based AI agents can replace traditional Static Application Security Testing (SAST) tools like Bandit. The study found that current general-purpose open-source models underperform specialized security tools, suggesting agentic AI is not yet ready for autonomous vulnerability detection in real-world conditions.
AINeutralarXiv – CS AI · Jun 106/10
🧠A comprehensive empirical study examines how German software engineers adopt generative AI tools, revealing that experience level, organizational size, and lack of project context awareness significantly influence effectiveness. The research combines 18 interviews with 109 survey responses to identify adoption patterns and barriers in a regulatory-constrained environment.
AINeutralarXiv – CS AI · Jun 56/10
🧠Researchers conducted interviews with 17 experienced developers to understand how they actually oversee autonomous software agents in practice, identifying four forms of oversight work (a priori control, co-planning, real-time monitoring, and post hoc review) and documenting practical challenges developers face when managing AI agents.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers introduce AblationBench, a benchmark suite for evaluating language model agents on ablation planning tasks in AI research. The study finds that frontier LMs achieve only 45% accuracy on average, significantly below human performance, highlighting challenges in automating scientific research methodologies.
🏢 Hugging Face
AINeutralarXiv – CS AI · Apr 146/10
🧠A large-scale empirical study of 679 GitHub instruction files shows that AI coding agent performance improves by 7-14 percentage points when rules are applied, but surprisingly, random rules work as well as expert-curated ones. The research reveals that negative constraints outperform positive directives, suggesting developers should focus on guardrails rather than prescriptive guidance.
AINeutralarXiv – CS AI · Apr 146/10
🧠Researchers conducted the first large-scale empirical analysis of AI decision-making across 366,120 responses from 8 major models, revealing measurable but inconsistent value hierarchies, evidence preferences, and source trust patterns. The study found significant framing sensitivity and domain-specific value shifts, with critical implications for deploying AI systems in professional contexts.
AINeutralarXiv – CS AI · Apr 146/10
🧠A research study analyzing 892 Reddit posts from cybersecurity forums reveals how security practitioners currently use, perceive, and adopt large language models in Security Operations Centers. While practitioners leverage LLMs for productivity gains in low-risk tasks, significant concerns about reliability, verification overhead, and security risks prevent broader autonomous deployment in critical security operations.
