ECE 663: Machine Learning in Adversarial Settings (Spring 2023)

---

Instructor

Neil Gong, neil.gong@duke.edu

Teaching Assistant

Hongbin Liu, hongbin.liu@duke.edu
Qitong Gao, qitong.gao@duke.edu

Lectures

Time: MW 3:30pm - 4:45pm
Location: Teer 106

Office Hours

Time: Thursday 1:00pm - 2:00pm
Location: 413 Wilkinson Building

Tentative Schedule

01/11    Course overview (Slides)

01/16    Holiday

01/18    Adversarial examples (white-box) (Slides)

• Towards Evaluating the Robustness of Neural Networks

01/23    Adversarial examples (black-box) (Slides)

• HopSkipJumpAttack: A Query-Efficient Decision-Based Attack
• Optional: Delving into Transferable Adversarial Examples and Black-box Attacks

01/25    Empirical defenses against adversarial examples (Slides)

• Towards Deep Learning Models Resistant to Adversarial Attacks
• Optional: Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Optional: Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods

01/30    Certified defenses against adversarial examples (Slides)

• Certified Adversarial Robustness via Randomized Smoothing

02/01    Data poisoning attacks to classifiers (Slides)

• Poisoning Attacks against Support Vector Machines
• Optional: Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networks

02/06    Data poisoning attacks to recommender systems (Slides)

• Toward trustworthy recommender systems: An analysis of attack models and algorithm robustness

02/08    Data poisoning attacks to graph-based methods (Slides)

• Attacking Graph-based Classification via Manipulating the Graph Structure
• Optional: Adversarial Attacks on Neural Networks for Graph Data

02/13    Model poisoning attacks to federated learning (Slides)

• Local Model Poisoning Attacks to Byzantine-Robust Federated Learning
• Speakers: Yixin Zhang and Pingcheng Jian

02/15    Certified defenses against data poisoning attacks (Slides)

• Intrinsic Certified Robustness of Bagging against Data Poisoning Attacks
• Optional: Certified Robustness of Nearest Neighbors against Data Poisoning Attacks
• Optional: Certified Defenses for Data Poisoning Attacks

02/20    Backdoor attacks to classifiers (Slides)

• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
• Optional: Trojaning Attack on Neural Networks
• Speakers: Haocheng Meng, Fakrul Islam Tushar, Yoo Bin Shin, and Yanting Wang

02/22    Backdoor attacks to pre-trained foundation models (Slides)

• BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning
• Optional: Poisoning and Backdooring Contrastive Learning
• Speakers: Yi Gao, Bofeng Chen, Zhicheng Guo, and Danyu Sun

02/27    Empirical defenses against backdoor attacks (Slides)

• Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks
• Optional: STRIP: A Defence Against Trojan Attacks on Deep Neural Networks
• Speakers: Haoming Yang, Oded Schlesinger, Rucha Patil, and Angikar Ghosal

03/01    Debugging data poisoning and backdoor attacks

• Identifying a Training-Set Attack's Target Using Renormalized Influence Estimation
• Optional: Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks

03/06    Model stealing attacks (Slides)

• Stealing Machine Learning Models via Prediction APIs
• Optional: Stealing Hyperparameters in Machine Learning
• Speakers: Minxue Tang, Yitu Wang, and Xueying Wu

03/08    Defenses against model stealing attacks

• Prediction Poisoning: Utility-Constrained Defenses Against Model Stealing Attacks
• Optional: PRADA: Protecting Against DNN Model Stealing Attacks

03/13    Spring recess

03/15    Spring recess

03/20    Intellectual property protection (Slides)

• Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring
• Optional: Certified Neural Network Watermarks with Randomized Smoothing
• Speakers: Minke Yu, Ziling Yuan, and Zhihao Dou

03/22    Privacy attacks - model inversion and membership inference (Slides)

• Membership Inference Attacks against Machine Learning Models
• Optional: Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures

03/27    Privacy attacks to federated learning (Slides)

• Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
• Optional: Deep Leakage from Gradients
• Speakers: Pragya Sharma, Tianyi Hu, and William Chown

03/29    Privacy-preserving machine learning via statistical methods (Slides)

• Deep Learning with Differential Privacy

04/03    Privacy-preserving machine learning via cryptographic methods (Slides)

• SecureML: A System for Scalable Privacy-Preserving Machine Learning

04/05    Data tracing in machine learning (Slides)

• Radioactive data: tracing through training

04/10    Misuse of machine learning (Slides)

• Private traits and attributes are predictable from digital records of human behavior

04/12    Deepfakes

• Generative Adversarial Nets
• Stable Diffusion
• ChatGPT
• Speakers: Grant Costa, Suhang Jiang, Martin Kuo, and Roger Chang

04/17    Project presentation

• Group 1: Fakrul Islam Tushar, Haocheng Meng, Yanting Wang, Yoo Bin Shin
• Group 2: Yixin Zhang, Pingcheng Jian
• Group 3: Roger Chang, Martin Kuo, Grant Costa, Suhang Jiang
• Group 4: Bofeng Chen, Yi Gao, Danyu Sun, Zhicheng Guo

04/19    Project presentation

• Group 5: Minke Yu, Ziling Yuan, Zhihao Dou
• Group 6: Minxue Tang, Yitu Wang, Xueying Wu
• Group 7: Haoming Yang, Oded Schlesinger, Rucha Patil, Angikar Ghosal
• Group 8: Tianyi Hu
• Group 9: William Chown, Pragya Sharma

Course Description

Machine learning is being widely deployed in many aspects of our society. Our vision is that machine learning systems will become a new attack surface and attackers will exploit the vulnerabilities in machine learning algorithms and systems to subvert their security and privacy. In this course, we will discuss security and privacy attacks to machine learning systems and state-of-the-art defenses against them.

Class Format

The class is paper reading, lecture, discussion, and project oriented. We will focus on one topic in a lecture. Students are expected to read the suggested papers on the topic and send comments to a specified email address by the end of the day before the lecture. Students are expected to lead a lecture on a chosen topic, complete a class project, present the project, and write a project report.

Group

Students can form groups of at most 4 students for the lecture and class project.

Deadlines

Reading assignments

• Sunday and Tuesday 11:59pm. Send comments to adversarialmlduke@gmail.com. Please send your comments to all papers in a single email thread.

Choosing a topic for lecture

• A group sends three preferred dates to adversarialmlduke@gmail.com by 11:59pm, 01/25.

Class project

• 02/01: project proposal due.
• 03/15: milestone report due.
• 04/17, 04/19: project presentation.
• 04/30: final project report due.

Grading Policy

50% project
25% reading assignment
10% class participation
15% class presentation