CourtGuard: A Model-Agnostic Framework for Zero-Shot Policy Adaptation in LLM Safety

Researchers introduce CourtGuard, a new framework for AI safety that uses retrieval-augmented multi-agent debate to evaluate LLM outputs without requiring expensive retraining. The system achieves state-of-the-art performance across 7 safety benchmarks and demonstrates zero-shot adaptability to new policy requirements, offering a more flexible approach to AI governance.

• →CourtGuard addresses the rigidity problem of current LLM safety mechanisms that require expensive retraining for new governance rules.
• →The framework uses an adversarial debate system grounded in external policy documents to evaluate AI safety without fine-tuning.
• →It achieved 90% accuracy on an out-of-domain Wikipedia Vandalism task by simply swapping reference policies, demonstrating zero-shot adaptability.
• →The system successfully curated and audited nine novel datasets of sophisticated adversarial attacks for automated data curation.
• →The approach decouples safety logic from model weights, providing a more interpretable and adaptable solution for AI governance.