DP-Hype: Federated Differentially Private Hyperparameter Search
Authors: Johannes Liebenow (University of Lübeck), Thorsten Peinemann (University of Lübeck), Esfandiar Mohammadi (University of Lübeck)
Volume: 2026
Issue: 3
Pages: 139–159
DOI: https://doi.org/10.56553/popets-2026-0075
Abstract: The tuning of hyperparameters in federated machine learning can substantially impact model performance. When the hyperparameters are tuned on sensitive data, privacy becomes an important challenge and to this end, differential privacy has emerged as the de facto standard for provable privacy. A standard setting when performing federated learning tasks is that clients agree on a shared setup, i.e., find a compromise from a set of hyperparameters, like a model's learning rate. Yet, prior work on privacy-preserving hyperparameter tuning is tailored to specific learning tasks, does not take into account the privacy leakage of aggregated results, or offers a sub-optimal privacy-utility trade-off. In this work, we present our algorithm DP-Hype, which performs a federated and privacy-preserving hyperparameter search by conducting a federated voting based on local hyperparameter evaluations of clients. In this way, DP-Hype selects hyperparameters that lead to a compromise supported by the majority of clients, while maintaining scalability and independence from specific learning tasks. We prove that DP-Hype preserves the strong notion of differential privacy called client-level differential privacy and, importantly, show that its privacy guarantees do not depend on the number of hyperparameters. We also provide bounds on its utility guarantees, that is, the probability of finding good hyperparameters, and implement DP-Hype as a submodule in the popular Flower framework for federated machine learning. In addition, we evaluate performance on multiple benchmark data sets in iid as well as multiple non-iid settings and demonstrate high utility of DP-Hype even under small privacy budgets.
Keywords: Differential Privacy, Federated Learning, Hyperparameters, Privacy-Preserving Machine Learning
Copyright in PoPETs articles are held by their authors. This article is published under a Creative Commons Attribution 4.0 license.
