Model Collapse Is Not a Bug but a Feature in Machine Unlearning for LLMs

Researchers propose Partial Model Collapse (PMC), a novel machine unlearning method for large language models that removes private information without directly training on sensitive data. The approach leverages model collapse - where models degrade when trained on their own outputs - as a feature to deliberately forget targeted information while preserving general utility.

• →PMC introduces a new unlearning approach that doesn't require sensitive data in the training objective, reducing privacy risks.
• →The method exploits model collapse phenomenon where training on self-generated outputs leads to information removal.
• →PMC overcomes four key limitations of existing unlearning methods that explicitly optimize on removal targets.
• →The approach demonstrates better preservation of general model utility compared to traditional unlearning techniques.
• →This represents a significant advancement in privacy-preserving AI that aligns with real-world data protection constraints.