SPARE: Self-distillation for PARameter-Efficient Removal

Researchers introduce SPARE, a new machine unlearning method for text-to-image diffusion models that efficiently removes unwanted concepts while preserving model performance. The two-stage approach uses parameter localization and self-distillation to achieve selective concept erasure with minimal computational overhead.

• →SPARE addresses the challenging problem of machine unlearning in text-to-image diffusion models with reduced computational costs.
• →The method uses gradient-based saliency to identify parameters responsible for unwanted concepts and constrains updates through sparse low-rank adapters.
• →A self-distillation objective overwrites unwanted concepts with user-defined surrogates while preserving other model behaviors.
• →SPARE outperforms current state-of-the-art on the UnlearnCanvas benchmark with fine-grained control over forgetting-retention trade-offs.
• →The approach enables compliance with data protection regulations and responsible AI practices in generative models.