Partition-Losses Fine-Tuning: Contamination-Robust Backdoor Unlearning

The widespread use of large-scale, weakly curated training data and third-party checkpoints makes training convenient, but leaves room for poisoning-based backdoor attacks. These attacks embed a backdoor through data poisoning in the training set: the infected model behaves normally on clean inputs but predicts an attacker-chosen label whenever the trigger appears, posing risks for security-sensitive deployment and model reuse. Post-training fine-tuning has become a practical default defense as it is computationally efficient and does not require control over the original training pipeline. However, most existing post-training fine-tuning defenses rely exclusively on a clean dataset and discard or avoid suspicious inputs, leaving potentially useful information unexploited. As a result, such clean-only approaches optimize benign performance alone and do not directly address the trigger-to-target association. In this paper, we propose a simple and architecture-agnostic post-training method, called Partition-Losses Fine-Tuning (PL), that leverages both clean data and previously overlooked suspicious samples. PL simultaneously encourages correct predictions on clean inputs while discouraging attacker-specified predictions on suspicious inputs, directly breaking the trigger-to-target association. Comprehensive experiments show that PL matches or surpasses clean-only fine-tuning methods under the same computational budget while substantially reducing clean-data requirements, and remains effective under realistic contamination, hyperparameter variation, and cross-attack settings.