Abstract: Network-level adversaries have been developing increasingly sophisticated techniques to perform surveillance and exert control over user traffic. We present a novel Connection Migration Powered Splitting (CoMPS) framework to construct multiple new defenses against various traffic analysis attacks. CoMPS limits the amount of information a particular adversary can observe on the network by performing traffic splitting within individual sessions. CoMPS is the first to fully support mid-session traffic splitting across heterogeneous network paths and protocols, without the need for deploying additional network infrastructure. CoMPS is not only readily deployable with any protocol supporting connection migration (e.g., QUIC, WireGuard, and Mosh), but incurs very little overhead.

We implement a working prototype of CoMPS and use CoMPS to develop a novel defense against website fingerprinting attacks. To evaluate the effectiveness of our defense, we use both simulated splitting data and web traffic that is split real-time using our prototype. Our defense outperforms other state-of-the-art web fingerprinting defenses against a powerful, adaptive adversary, while incurring smaller overhead (decreasing throughput by just 7%). We also propose this framework for other network privacy use cases, such as censorship circumvention.

Abstract: It is known that deep neural networks, trained for the classification of a non-sensitive target attribute, can reveal sensitive attributes of their input data; through features of different granularity extracted by the classifier. We, taking a step forward, show that deep classifiers can be trained to secretly encode a sensitive attribute of users' input data, at inference time, into the classifier's outputs for the target attribute. An attack that works even if users have a white-box view of the classifier, and can keep all internal representations hidden except for the classifier's estimation of the target attribute. We introduce an information-theoretical formulation of such adversaries and present efficient empirical implementations for training honest-but-curious (HBC) classifiers based on this formulation: deep models that can be accurate in predicting the target attribute, but also can utilize their outputs to secretly encode a sensitive attribute. Our evaluations on several tasks in real-world datasets show that a semi-trusted server can build a classifier that is not only perfectly honest but also accurately curious. Our work highlights a vulnerability that can be exploited by malicious machine learning service providers to attack their user's privacy in several seemingly safe scenarios; such as encrypted inferences, computations at the edge, or private knowledge distillation. We conclude by showing the difficulties in distinguishing between standard and HBC classifiers and discussing an extension of this attack to a more general setting where, by allowing a few more queries, an attacker cannot only infer a sensitive attribute, but it also can (approximately) reconstruct the whole private input.

Abstract: End-to-end encryption (E2EE) in messaging platforms enable people to securely and privately communicate with one another. Its widespread adoption has however raised concerns that illegal content might now be shared undetected. Following the global pushback against key escrow systems, client-side scanning based on perceptual hashing has been recently proposed by governments and researchers to detect illegal content in E2EE communications. In this talk, we will present what is to the best of our knowledge the first framework to evaluate the robustness of perceptual hashing-based client-side scanning. We will describe a new class of detection avoidance attacks and show current systems to not be robust.

More specifically, we will present a general black-box attack against any perceptual hashing algorithm and two white-box attacks for discrete cosine-based algorithms. We show perceptual hashing-based client-side scanning mechanisms to be highly vulnerable to detection avoidance attacks in a black-box setting. We show in a large-scale evaluation that more than 99.9% of images can be successfully attacked while preserving the content of the image. Furthermore, we show our attack to generate diverse perturbations, strongly suggesting that straightforward mitigation strategies would be ineffective. Finally, we show that the larger thresholds necessary to make the attack harder would probably require more than one billion images to be flagged and decrypted daily, raising strong privacy concerns. Taken together, our results shed serious doubts on the robustness of perceptual hashing-based client-side scanning mechanisms currently proposed by governments, organizations, and researchers around the world.

Abstract: Sometimes ads and analytics libraries behave more in line with malware, doing things like actively circumventing operating system protections, doing home network scans, and encrypting all their strings to be decrypted at runtime. And these libraries are included in apps with hundreds of millions of installs! In this talk, we'll look at sample set of these more extreme behaviours we've found over time and look at what they are doing, and the steps we followed to actually figure out what they are doing.

Abstract: The California Consumer Privacy Act (CCPA)---which began enforcement on July 1, 2020---grants California users the affirmative right to opt-out of the sale of their personal information. In this work, we perform a series of observational studies to understand how websites implement this right and how this implementation has evolved over the first year. We perform manual analyses of the top 500 U.S. websites and classify how each site implements this new requirement; e also perform automated analyses of the Top 5000 U.S. websites. We find that the vast majority of sites that implement opt-out mechanisms do so with a Do Not Sell link rather than with a privacy banner, and that many of the linked opt-out controls exhibit features such as nudging and indirect mechanisms (e.g., fillable forms). We then perform a pair of user studies with 4357 unique users (recruited from Google Ads and Amazon Mechanical Turk) in which we observe how users interact with different opt-out mechanisms and evaluate how the implementation choices we observed---exclusive use of links, prevalent nudging, and indirect mechanisms---affect the rate at which users exercise their right to opt-out of sale. We find that these design elements significantly deter interactions with opt-out mechanisms---including reducing the opt-out rate for users who are uncomfortable with the sale of their information---and that they reduce users' awareness of their ability to opt-out. Our results demonstrate the importance of regulations that provide clear implementation requirements in order empower users to exercise their privacy rights.

Abstract: When law enforcement agencies (LEAs) ask for backdoors in end-to-end encryption systems, most information security professionals' reaction is wholesale rejection. This pushes law LEAs to use zero-day exploits instead, which is harmful to security overall. Perhaps it would be better to have an explicit backdoor mechanism that ensures accountability, and which has safeguards to prevent it being used for mass surveillance.

I propose the following: a provider of a communication service (e.g. Facebook) maintains a publicly readable transparency log, similar to Certificate Transparency, containing all of the law enforcement intercept orders they have received and accepted. Each log entry contains the jurisdiction of the warrant, a code indicating the reason (terrorism, child sexual abuse, etc.), validity start and end date, and a cryptographic commitment to a single device ID that is the target of the warrant. Thus, anybody can see how many warrants are being issued in which jurisdiction and for which reason, but not who their targets are.

To intercept a device, the communication service provider must first add the entry to the log, then send a message to the device that reveals the device ID in the commitment, and a proof that the entry is included in the log. The software on the user's device checks whether the log entry is for its own device ID, and if it is valid, the software silently uploads a cleartext copy of the requested data to the appropriate LEA. This upload feature is essentially identical to the cloud backup feature that is already built into otherwise encrypted messaging apps such as WhatsApp and iMessage; the only effect of the backdoor is to enable this backup, even if it had been disabled by the user.

Additionally, in each jurisdiction there is a trusted oversight board. The service provider reveals the target of each log entry to the oversight board in the appropriate jurisdiction; the board checks that each log entry has a corresponding warrant, and that the warrant is genuine and legal. If the board determines that the system is being abused, it has legal powers to stop it.

Unlike key escrow and other backdoor proposals, this approach ensures the backdoor cannot be used without leaving a public audit trail, and it does not involve any weakening of the cryptographic protocols. There is no single "golden key" that can decrypt all communications. Service providers are forced to be explicit about the jurisdictions in which they will accept warrants. The number of targeted users is public, which gives us reassurance that the system is not being used for mass surveillance.

Moreover, the system is simple enough that non-technical people can understand it. It places more faith in established democratic structures (e.g. the judiciary and our democratically elected representatives) and less trust in unaccountable tech companies. LEAs have a democratic mandate to investigate crimes, and I believe this proposal enables LEAs to do their job, while also protecting the civil liberties that form the foundation of a democratic society.