Explainable LLM Unlearning Through Reasoning

Researchers introduce Targeted Reasoning Unlearning (TRU), a new method for removing specific knowledge from large language models while preserving general capabilities. The approach uses reasoning-based targets to guide the unlearning process, addressing issues with previous gradient ascent methods that caused unintended capability degradation.

• →TRU addresses safety, copyright, and privacy concerns by enabling more precise knowledge removal from LLMs.
• →Previous gradient ascent methods caused unintended degradation of general capabilities and incomplete knowledge removal.
• →The new reasoning-based unlearning target provides explicit guidance on what and how models should unlearn.
• →TRU combines cross-entropy supervised loss with gradient ascent-based loss for more targeted unlearning.
• →The method shows superior robustness under diverse attack scenarios while maintaining general model capabilities.