ICCU: In-Context Continual Unlearning via Pattern-Induced Refusal Rules

Researchers introduce ICCU, an in-context continual unlearning framework that removes specific data influence from language models without modifying parameters. The method uses pattern-induced refusal rules applied at inference time, addressing the inefficiency of sequential unlearning requests in production deployments.

ICCU represents a significant advancement in machine unlearning, tackling a critical problem in deploying large language models at scale. Traditional fine-tuning approaches to unlearning prove costly and problematic when handling multiple sequential requests, accumulating utility loss and creating interference between different unlearning tasks. This research proposes a fundamentally different approach: instead of retraining models, ICCU generates readable refusal rules from unlearning datasets and applies them during inference through filtering or system prompts.

The development addresses growing regulatory and ethical pressures around data privacy in AI systems. As jurisdictions implement stronger data protection requirements and users demand the ability to remove their data from trained models, production systems need efficient unlearning mechanisms. Current methods require expensive retraining cycles, making compliance costly and operationally challenging.

The framework's key innovation lies in its compositional design. Rules accumulate as an order-independent union, meaning new unlearning requests don't interfere with previous ones and don't require reprocessing historical data. This enables scalable, stateless unlearning that preserves model utility while effectively suppressing target knowledge. The research demonstrates robustness against paraphrased and cross-lingual variations of suppressed content, suggesting practical resilience against evasion attempts.

For developers and enterprises deploying language models, ICCU could substantially reduce operational costs and complexity around data removal requests. The ability to handle unlearning without parameter updates means no downtime, no model versioning issues, and immediate deployment. As AI regulation intensifies globally, efficient unlearning mechanisms become competitive advantages, making this architectural approach increasingly valuable for production systems managing privacy compliance at scale.

• →ICCU enables unlearning without model retraining, using inference-time refusal rules that preserve original model parameters.
• →The order-independent rule accumulation prevents cross-request interference, allowing sequential unlearning requests to scale efficiently.
• →Forget-set data can be discarded after rule induction, reducing storage requirements and privacy risks in production systems.
• →The framework demonstrates robustness against paraphrased and cross-lingual query variations, addressing practical evasion scenarios.
• →Parameter-free unlearning enables immediate deployment without model versioning, downtime, or utility loss in production environments.