Adversarial machine learning tools benchmarker

Use

1. download images (./download_images.sh)
2. download model checkpoints (./download_model_checkpoints.sh)
3. run attacks (./run_attacks.sh)
4. run defenses (./run_defenses.sh)
5. compute perturbation magnitude (./computer_perturbation_magnitude.sh)

Supported attacks

Evasion Attacks:

• Fast gradient method (FGM) (Goodfellow et al., 2014)
• DeepFool (Moosavi-Dezfooli et al., 2015)
• Basic iterative method (BIM) (Kurakin et al., 2016)
• Carlini & Wagner (C&W) (Carlini and Wagner, 2016)
• Projected gradient descent (PGD) (Madry et al., 2017)
• Jacobian saliency map (Papernot et al., 2016)

Supported tools

• CleverHans
• Foolbox
• Adversarial Robustness 360 Toolbox

Supported data sets

• ImageNet
• Cifar10
• MNIST

Supported models

• Inception V3 (Szegedy et al.)
• Wide Resnet (Zagoruyco and Komodakis)
• Madry et al.

Dependencies

• cleverhans 3.0.1
• foolbox 2.3.0
• adversarial-robustness-toolbox 1.0.1
• tensorflow 1.15.0
• tensorflow-datasets 1.3.2
• pandas 0.25.0
• numpy 1.18.0