Weight initialization based on gradient similarity for versatile machine unlearning
Authors: Doun Lee (Kookmin University), Jongyun Shin (Kookmin University), Jinwoo Bae (Kookmin University), Hyunjoon Cho (Kookmin University), Jangho Kim (Kookmin University)
Volume: 2026
Issue: 3
Pages: 575–603
DOI: https://doi.org/10.56553/popets-2026-0097
Abstract: The growing necessity for deep learning applications to adhere to rising data privacy standards has made machine unlearning crucial in removing the impact of specific examples from a given model. Although exact unlearning, which retrains the model from scratch using the remaining dataset, satisfies this objective, it is computationally expensive, leading approximate unlearning to be an active research field. However, we argue that most existing approximate algorithms fail to provide robust privacy guarantees. These methods are often "biased," designed to defend against attacks on a single model property (e.g., loss) while leaving information encoded in other properties (e.g., entropy) exposed. An adaptive attacker can simply exploit this bias, making the unlearning ineffective, as a model's privacy is only as strong as its most vulnerable point. For example, gradient descent using random labels may enhance indistinguishability with respect to loss by directly lowering the probability of the correct label, but it fails to achieve comparable results for entropy, as it does not increase the entropy to levels similar to those of unseen data. To address this problem, we propose a Weight Initialization based on Gradient similarity dubbed WIG, a novel algorithm that provides unbiased unlearning that approximates the retraining process. Instead of targeting a specific property, WIG induces catastrophic forgetting by partially initializing weights based on their gradient similarity between the train set and the data to be forgotten. WIG achieves the best indistinguishability among current state-of-the-art approximate unlearning algorithms across diverse metrics, including various membership inference attacks, Inter-class confusion test, U-LiRA, NeurIPS Machine Unlearning Challenge, and Gaussian Poison, demonstrating its versatility.
Keywords: Machine learning, Machine unlearning, Deep learning, Data privacy
Copyright in PoPETs articles are held by their authors. This article is published under a Creative Commons Attribution 4.0 license.
