HyDia:FHE-based Facial Matching with Hybrid Approximations and Diagonalization
Authors: Sam Martin (University of Notre Dame), Nirajan Koirala (University of Notre Dame), Helena Berens (University of Notre Dame), Thomas Rozgonyi (University of Notre Dame), Micah Brody (University of Notre Dame), Taeho Jung (University of Notre Dame)
Volume: 2025
Issue: 4
Pages: 585–604
DOI: https://doi.org/10.56553/popets-2025-0146
Abstract: Secure facial matching systems play a crucial role in privacy preserving biometric authentication, particularly in domains such as law enforcement, border control, and healthcare. Traditional facial matching systems require direct access to biometric data, raising significant privacy concerns. This paper presents HyDia, a novel protocol for scalable FHE-based facial matching with high computation and communication efficiencies, enabling secure one-to-many facial matching without exposing biometric data in plaintext. Our protocol adapts diagonalized matrix multiplication techniques to accommodate highly imbalanced matrix computations, enabling our novel non-rotational inner product algorithm that substantially reduces the homomorphic computation overhead compared to prior works. We further propose a hybrid approximation method for homomorphic thresholding, which achieves better approximation than the state-of-the-art approach (Chebyshev approximation) at the same multiplicative depths. More importantly, our design does not reveal exact similarity scores to the querier; instead, it provides only a threshold-based match decision or matching sources, strengthening privacy by withholding granular database information. We implement HyDia and competing approaches and provide both formal security proof and extensive experimental validation. Our results show that HyDia achieves practical query times at scale, significantly outperforming existing HE-based solutions in both computation and communication overhead. Notably, HyDia is the only viable FHE-based approach in common bandwidth settings (2Mbps & 1Gbps), outperforming the state-of-the-art approaches by 5.2x-227.4x in end-to-end latency under different settings. Finally, our experiments on real-face datasets show that HyDia incurs negligible accuracy loss, by achieving the same F1 score of 0.9968 as the corresponding plaintext facial matching baselines. This work advances the feasibility of privacy-preserving biometric identification, offering a scalable, bandwidth-efficient, and accurate solution for real-world deployments.
Keywords: Facial Matching, Biometric Security, FHE, CKKS
Copyright in PoPETs articles are held by their authors. This article is published under a Creative Commons Attribution 4.0 license.
