'Your AI Text is not Mine': Redefining and Evaluating AI-generated Text Detection under Realistic Assumptions

Researchers have released AITDNA, a new benchmark dataset for detecting AI-generated text that includes detailed edit histories and human-machine co-creation information. The study reveals that existing AI text detectors perform inconsistently across different types of AI-generated content, highlighting the need for standardized definitions of what constitutes problematic AI-generated text and more robust detection methods.

The proliferation of large language models has created genuine concerns about AI-generated text's societal impact, yet the field lacks coherent standards for defining and detecting such content. This research addresses a fundamental gap in the AI detection landscape by systematizing what 'AI-generated text' actually means in practice. Rather than relying on binary classifications, the researchers recognize that AI involvement exists on a spectrum—from fully machine-generated content to minimally edited human work—requiring nuanced detection approaches.

The introduction of AITDNA represents a methodological advancement because it captures the full interaction history between humans and AI systems, reflecting real-world usage patterns. Most existing datasets oversimplify detection scenarios by assuming clean separation between human and machine text. The benchmark's emphasis on genesis information acknowledges that detection difficulty increases substantially when humans iteratively refine AI outputs or when AI enhances human-written content.

For developers building detection tools, the findings carry significant implications. Current detectors optimized for specific notions of AI generation fail when deployment scenarios shift. This fragmentation undermines confidence in detection systems used for academic integrity, content authenticity, and misinformation prevention. The research demonstrates that detector performance remains inconsistent across different AI models, editing patterns, and text types.

The public release of code and data enables standardized evaluation moving forward. As AI text generation becomes more sophisticated and human-AI collaboration becomes standard practice, the field must develop detectors that perform reliably across varied realistic scenarios rather than narrow, idealized conditions. This research establishes foundational infrastructure for that evolution.

• →Existing AI text detectors perform well only for specific definitions of AI-generated content, not as universal solutions.
• →AITDNA benchmark includes complete edit and interaction histories, capturing realistic human-machine collaboration patterns.
• →The field lacks consensus on what constitutes harmful AI-generated text, leading to inconsistent detection approaches.
• →Current detection methods fail when AI involvement exists on a spectrum rather than as binary human-or-machine classification.
• →Standardized benchmarks with detailed genesis information are essential for developing robust detection systems.