GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks

Researchers introduce GenTI, an LLM-driven framework that automatically generates intrusion detection and prevention system (IDPS) rules for zero-day and unseen attacks. The benchmark dataset aggregates over 150,000 Snort/Suricata rules and 50,000 YARA signatures with structured cybersecurity intelligence, achieving 87.4% detection accuracy on unseen threats while reducing false positives from 8.5% to 2.3%.

The GenTI framework addresses a critical gap in cybersecurity automation by leveraging large language models to synthesize detection rules without manual expert intervention. Traditional IDPS systems rely on manually crafted signatures that struggle against emerging threats and zero-day exploits, creating operational bottlenecks for security teams. GenTI transforms this constraint by enabling LLM-driven rule generation through structured prompt engineering and Chain-of-Thought reasoning, paired with a Chain-of-Verification validation loop that ensures syntactic correctness and security effectiveness.

The research builds on growing recognition that AI can accelerate security operations. Previous datasets like CICIDS2017 and UNSW-NB15 focused primarily on traffic classification without the structured annotations necessary for automated rule synthesis. GenTI's dataset enriches over 150,000 rules with protocol behaviors, payload signatures, and Cyber Threat Intelligence mappings, creating the infrastructure for scalable rule generation.

The benchmark's performance metrics carry operational significance. Achieving 87.4% detection on unseen attacks represents substantial improvement over manual approaches, while 2.3% false-positive rates directly reduce alert fatigue that degrades analyst productivity. The 94.8% CTI coverage demonstrates that generated rules align with documented threat intelligence, enhancing organizational threat awareness.

The framework's implications extend beyond individual deployments. Self-evolving IDPS systems powered by GenTI could fundamentally reshape security team workflows by automating the most time-consuming detection engineering task. However, deployment requires careful validation that generated rules match organizational security postures and don't introduce compliance risks.

• →GenTI achieves 87.4% detection rate on unseen attacks, improving baseline detection from 45% through LLM-driven rule generation
• →The GTI dataset contains 150,000+ annotated IDPS rules with CTI mappings, creating standardized training data for security automation
• →False-positive rates drop to 2.3% through Chain-of-Verification validation, directly improving security analyst efficiency
• →Framework demonstrates 89.4% composite rule-quality score with 94.8% CTI coverage across generated signatures
• →LLM-based IDPS automation enables adaptive security systems that evolve against emerging threats without manual rule crafting