FIT to Forget: Robust Continual Unlearning for Large Language Models

Researchers introduce FIT, a continual unlearning framework enabling large language models to efficiently forget privacy-sensitive, copyrighted, and harmful content across sequential deletion requests. The method addresses critical limitations of existing single-shot unlearning approaches by preventing catastrophic forgetting while maintaining model utility, demonstrated across models up to 14B parameters.

The paper addresses a significant gap in LLM safety and compliance infrastructure. Current unlearning methods handle isolated deletion requests effectively but fail when processing deletion streams—a realistic scenario as regulatory frameworks like the EU's right to be forgotten and GDPR increasingly mandate content removal. FIT's three-mechanism approach (redundancy filtering, importance-aware algorithm selection, and targeted layer attribution) represents a technical advancement that acknowledges the fundamental tension between forgetting specific information and retaining general capabilities.

This work emerges amid growing regulatory pressure and legal challenges facing LLM developers. Companies deploying models at scale face mounting demands to remove memorized training data, yet naive deletion approaches cause performance degradation that threatens commercial viability. The introduction of PCH—a unified benchmark spanning personal, copyrighted, and harmful content categories—provides standardization previously lacking in unlearning research, enabling meaningful comparisons across methods.

For the AI industry, FIT's efficacy holds substantial implications. Organizations can now theoretically comply with deletion requests without accepting the computational cost of model retraining. The framework's demonstrated resilience against relearning and quantization recovery attacks addresses security concerns about unlearning reversibility. This technological progress directly supports the business case for deploying LLMs in regulated sectors including healthcare, finance, and legal services where data retention constraints are non-negotiable.

Key questions remain regarding scalability to trillion-parameter models and the framework's performance under adversarial conditions. Enterprise adoption depends on establishing whether FIT's benefits persist across the model sizes and training data diversity present in production systems. Continued development in this direction could reshape deployment timelines for compliant AI systems.

• →FIT enables LLMs to process continuous unlearning requests without performance degradation or catastrophic forgetting across sequential deletions
• →The PCH benchmark provides standardized evaluation metrics (Forget Degree and Retain Utility) for rigorous unlearning assessment across content categories
• →Framework maintains downstream task performance (GSM8K, MMLU) even after hundreds of sequential deletion requests
• →Resilience against relearning and quantization recovery attacks addresses critical security vulnerabilities in unlearning systems
• →Technical advancement supports regulatory compliance strategies for enterprise LLM deployment in privacy-sensitive domains