Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study

A comprehensive empirical study examined how developers use rules in AI-powered IDEs to constrain LLM behavior, extracting 7,310 rules from 83 open-source projects. The research revealed a significant gap between what developers prioritize (architectural constraints) and what they actually implement (low-level formatting rules), while showing that rule updates improve artifact compliance by an average of 23 percentage points.

This study addresses a critical blind spot in the rapidly expanding AI IDE ecosystem. As developers increasingly rely on Large Language Models to generate code, the mechanisms for injecting human intent and project-specific constraints have emerged as essential infrastructure—yet remained largely unstudied. The research bridges theory and practice by combining quantitative analysis of 7,310 real-world rules with qualitative insights from 99 practitioners, revealing how this novel software artifact actually functions in production environments.

The disconnect between stated priorities and observed behavior has significant implications. Developers recognize that architectural constraints matter most, yet their rule repositories emphasize low-level workflow automation and code formatting. This mismatch suggests either a gap in tool capabilities, insufficient guidance on best practices, or constraints on what can be effectively specified. The finding that rules evolve frequently through constructive expansions rather than refinements indicates developers treat rules as evolving scaffolding rather than static policies.

The compliance improvement metric—a 23 percentage point increase following rule updates—demonstrates concrete business value. This empirical validation matters for tool adoption and justifies the overhead of maintaining rules. However, the study exposes a fundamental tension: developers primarily update rules reactively to fix AI errors through negative constraints rather than proactively establishing comprehensive guidelines. This reactive posture may limit the full potential of rules as architectural governance mechanisms.

Future iterations of AI IDEs should prioritize automated conflict detection and context management, as the research recommends. The field needs better tooling to help developers translate high-level architectural intent into effective rule specifications, transforming rules from reactive patches into proactive design instruments.

• →Developers prioritize architectural constraints but predominantly implement low-level formatting rules in AI IDE configurations
• →Rule updates increase artifact compliance by 22.99% on average, providing empirical validation of their effectiveness
• →Most developers modify rules reactively by adding negative constraints to fix errors rather than proactively designing comprehensive guidelines
• →Repository analysis shows rule evolution is driven by constructive context expansions (29.17%) and enrichments (26.59%)
• →Tool builders should focus on automated conflict detection and context management mechanisms to optimize prompting strategies