This repository is part of a solution for the informatiCup 2019 competition hosted by the Gesellschaft für Informatik (GI). The task of the 14th informatiCup is to generate adversarial examples for a given neural network based classification API.
Our solution in this repository tries to solve the use case:
- The attacker has access to the dataset used to train the API classifier
- The API classifier only returns the top-5 results (class label and confidence)
- He wants to keep the number of necessary API queries as low as possible
We solved it by using an Evolutionary Algorithm (EA) in combination with an ensemble of local trained models assuming that a generated fooling image also fools another network.
The theoretical background, implementation details and results are highlighted in our short paper. To reproduce the results of our black-box analysis you can use the raw data (test_images_analysis_full.csv) and the respective analysis script analyze_blackbox.py.
Python3.6 is required, which is the default python version in Ubuntu 18.04 LTS. In order to execute this tool on a target model that is provided by the GI, an API key is required and has to be specified in config.json.
cd informatiCup2019-LabWITT-EA
sudo apt install python3-venv python3-tk python3-dev
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txtMake sure you are in the previously created virtual environment, indicated by (venv) at the beginning of your bash. (Enter the virtual environment with source venv/bin/activate and leave it by typing deactivate)
Basic approach, random image, e.g. class index 2.
python3 __main__.py -c 2 -l
Uses random polygons with 3-5 edges to create the fooling image, e.g. class index 2.
python3 __main__.py -l -p -c 2
Some classes are hard to fool, since they are most likely not in the top-5 result from a random image. Therefore a fooling image for a stop sign can be created by starting off with a stop sign. The input image has to be 64x64 and should be in the top 5 results of the target network.
python3 __main__.py -o path-to-your-png -l -c 2
A gray fooling image with polygons is created, which class it fools is not further specified (-1).
python3 __main__.py -p -g -l -c -1
The fooling image is stored in results. Additionally a GIF showing the image evolution is stored and a minimal graph plots the confidence over the iterations.
| Option | Type | Description |
|---|---|---|
| ['-c', '--classes'] | Integer | Space separated class label list, indicating for which classes a fooling image should be produced |
| ['-l', '--local_models'] | Flag | Use local models for fooling |
| ['-g', '--grayscale'] | Flag | Generate grayscale images |
| ['-p', '--poly'] | Flag | Use polygons instead of random pixels |
| ['-o', '--original'] | Image Path | Input image as startup aid |
| [-s, '--statistic'] | Flag | Save statistic to file |
A config file contains some more values which can be changed, but there is no need to do it. E.g. changing the minimum confidence of a fooling image.
By default, two local networks are used for our approach. If you want to train your own models, check out our repo: https://github.com/MaximilianIdahl/gtsrb-models-keras.
- Develop a web-based application of this tool
- Support to set local models easy as target
- Integrate the functionality of the CPPN repository (but only support torch?)