Example Notebooks¶
HEART Library Notebooks can be found on github. Notebooks beginning with ‘get_started’ are for all T&E users and are a great place to start.
0_get_started_heart.ipynb¶
Description: This notebook demonstrates how to get started with HEART
Intended Audience: All T&E users
1_get_started_pgd_attack.ipynb¶
Description: This notebook quickly demonstrates the use of the Projected Gradient Descent attack in code as part of Test & Evaluation of small benchmark datasets (MNIST and CIFAR-10). Key features:
Attacks are demonstrated using parallel and non-parallel mode.
Extracting attack metadata to determine the best performing attacks.
Calculating clean and robust accuracy
Calculating perturbation
Intended Audience: All T&E users
2_get_started_auto_attack.ipynb¶
Description: This notebook quickly demonstrates the use of AutoAttack in code as part of Test & Evaluation of small benchmark datasets (MNIST and CIFAR-10). Key features:
Attacks are demonstrated using parallel and non-parallel mode.
Extracting attack metadata to determine the best performing attacks.
Calculating clean and robust accuracy
Calculating perturbation
Intended Audience: Advanced T&E users
3_template_for_advanced_T&E.ipynb¶
Description: This notebook walks through an advanced T&E example for an image classification task.
Define the T&E task
Set up the dataset
Load the classification model
Scope relevant attacks
Execute evaluation
Draw conclusions on model vulnerabilities
Next steps
Intended Audience: Advanced T&E Users
4_get_started_adversarial_patch.ipynb¶
Description: This notebook demonstrates how to perform an adversarial patch attack on CIFAR-10 data and a simple classification model using HEART and MAITE
Intended Audience: All T&E Users
5_get_started_object_detection.ipynb¶
Description: This notebook demonstrates how to perform object detection with DeTR using HEART and MAITE
Intended Audience: All T&E Users
6_adversarial_patch_for_object_detection.ipynb¶
Description: This notebook demonstrates how to perform adversarial patch attacks on object detection models using HEART and MAITE. For the purposes of this demonstration, DeTR will be the object detection model of choice and a variety of images from the COCO data set will be used for executing detection and attack.
Intended Audience: Advanced T&E Users
7_get_started_black_box_attacks.ipynb¶
Description: This notebook demonstrates how to perform black-box attacks using HEART and MAITE
Intended Audience: All T&E Users
8_get_started_defenses.ipynb¶
Description: This notebook demonstrates how to deploy defenses using HEART and MAITE
Intended Audience: All T&E Users
9_get_started_maite_evaluate.ipynb¶
Description: This notebook demonstrates how to execute evaluations using HEART and MAITE’s
`evaluate`functionIntended Audience: All T&E Users