#empirical-research News & Analysis

3 articles tagged with #empirical-research. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AINeutralarXiv – CS AI · 14h ago6/10

🧠

Do Agent Rules Shape or Distort? Guardrails Beat Guidance in Coding Agents

A large-scale empirical study of 679 GitHub instruction files shows that AI coding agent performance improves by 7-14 percentage points when rules are applied, but surprisingly, random rules work as well as expert-curated ones. The research reveals that negative constraints outperform positive directives, suggesting developers should focus on guardrails rather than prescriptive guidance.

AINeutralarXiv – CS AI · 14h ago6/10

🧠

Measuring the Authority Stack of AI Systems: Empirical Analysis of 366,120 Forced-Choice Responses Across 8 AI Models

Researchers conducted the first large-scale empirical analysis of AI decision-making across 366,120 responses from 8 major models, revealing measurable but inconsistent value hierarchies, evidence preferences, and source trust patterns. The study found significant framing sensitivity and domain-specific value shifts, with critical implications for deploying AI systems in professional contexts.

AINeutralarXiv – CS AI · 14h ago6/10

🧠

Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit

A research study analyzing 892 Reddit posts from cybersecurity forums reveals how security practitioners currently use, perceive, and adopt large language models in Security Operations Centers. While practitioners leverage LLMs for productivity gains in low-risk tasks, significant concerns about reliability, verification overhead, and security risks prevent broader autonomous deployment in critical security operations.