Skip to content

Release v1.1.0

Latest
Latest
Compare
Choose a tag to compare
@dlmgary dlmgary released this 14 Nov 18:08
· 5 commits to main since this release
v1.1.0
411737e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

What's new in Counterfit v1.1.0

                              __            _____ __
      _________  __  ______  / /____  _____/ __(_) /_
     / ___/ __ \/ / / / __ \/ __/ _ \/ ___/ /_/ / __/
    / /__/ /_/ / /_/ / / / / /_/  __/ /  / __/ / /
    \___/\____/\__,_/_/ /_/\__/\___/_/  /_/ /_/\__/

                    Version: 1.1.0

We're thrilled to announce a new release version for Counterfit! 🚀

1. Counterfit as a package

  • Counterfit can now be used from within a Python script.
  • Access to all of the targets and attacks that the Counterfit CLI supports.
import counterfit
import counterfit.targets as cf_targets


digits_target = cf_targets.Digits()
digits_target.load()
cf_attack = counterfit.Counterfit.build_attack(digits_target, 'hop_skip_jump')
results = counterfit.Counterfit.run_attack(cf_attack)

2. Updates to Counterfit CLI

  • Counterfit CLI now automatically loads attacks and frameworks.
  • Two new sample targets that show you how to attack reinforcement learning. i.e. Cart Pole targets.
$ counterfit
                          __            _____ __
  _________  __  ______  / /____  _____/ __(_) /_
 / ___/ __ \/ / / / __ \/ __/ _ \/ ___/ /_/ / __/
/ /__/ /_/ / /_/ / / / / /_/  __/ /  / __/ / /
\___/\____/\__,_/_/ /_/\__/\___/_/  /_/ /_/\__/

                Version: 1.1.0

counterfit> list targets 
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                ┃ Model Type ┃ Data Type ┃ Input Shape   ┃ # Samples ┃ Endpoint                                             ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ cart_pole           │ closed-box │ tabular   │ (1080000,)    │ 0         │ cartpole_dqn_10000.pt.gz                             │
│ cart_pole_initstate │ closed-box │ tabular   │ (4,)          │ 0         │ cartpole_dqn_10000.pt.gz                             │
│ creditfraud         │ closed-box │ tabular   │ (30,)         │ 0         │ creditfraud/creditfraud_sklearn_pipeline.pkl         │
│ digits_keras        │ closed-box │ image     │ (28, 28, 1)   │ 0.        │ digits_keras/mnist_model.h5                          │
│ digits_mlp          │ closed-box │ image     │ (1, 28, 28)   │ 0         │ digits_mlp/mnist_sklearn_pipeline.pkl                │
│ movie_reviews       │ closed-box │ text      │ (1,)          │ 0.        │ movie_reviews/movie_reviews_sentiment_analysis.pt    │
│ satellite           │ closed-box │ image     │ (3, 256, 256) │ 0         │ satellite/satellite-image-params-airplane-stadium.h5 │
└─────────────────────┴────────────┴───────────┴───────────────┴───────────┴──────────────────────────────────────────────────────┘

counterfit> set_target satellite

satellite> set_attack hop_skip_jump 
[+] success:  Using fb58020f

satellite>HopSkipJump:fb58020f> show info
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Attack Field ┃ Description                                                                                                                                               ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Name         │ hop_skip_jump                                                                                                                                             │
│ Type         │ closed-box                                                                                                                                                │
│ Category     │ evasion                                                                                                                                                   │
│ Tags         │ image, tabular                                                                                                                                            │
│ Framework    │ art                                                                                                                                                       │
│ Docs         │ Implementation of the HopSkipJump attack from Jianbo et al. (2019). This is a powerful closed-box attack that only requires final class prediction, and   │
│              │ is an advanced version of the boundary attack. | Paper link: https://arxiv.org/abs/1904.02144                                                             │
└──────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

satellite>HopSkipJump:fb58020f> run
HopSkipJump:   0%|                                                                                                                                    | 0/1 [00:00<?, ?it/sFailed to draw a random image that is adversarial, attack failed.
HopSkipJump: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:04<00:00,  4.88s/it]
[+] success:  Attack completed fb58020f

3. Update to internal architecture

  • Internal Counterfit repository is updated to be more extensible and easier to understand.
  • Updated targets and attacks naming convention.
  • Adds framework for integration tests before pull-requests can be merged to main.

4. Support for Reinforcement Learning attacks

  • Counterfit leverages OpenAI gym to use different attacks on Reinforcement Learning (RL) targets such as Cart Pole.
  • Allow Windows and Linux users with a GUI to see the live attack on the Cart Pole target.
  • Counterfit can run in headless mode to attack Cart Pole target when no GUI available.
  • See demo example on the Hands-on Hacking of Reinforcement Learning Systems demo in our Counterfit Examples document.

The not so fine print: Please read!

Release v.1.1.0 of Counterfit is not backwards compatible with Counterfit v1.0.0. This is due to major changes in the architecture of the software and how Counterfit targets and attacks are loaded. The good news is that Counterfit v1.1.0 has fully ported all the previous attacks and targets.

To improve the readability of the Counterfit command line interface (CLI), Counterfit v1.1.0 updates the naming convention of some of the commands while it deprecates others.

  • The interact command has been replaced with the more idiomatic set_target command.
  • The use command has been replaced with the more idiomatic set_attack command.
  • The set command has been replaced with the more idiomatic set_params command.
  • The load command has been deprecated. Frameworks and attacks are now automatically loaded by the CLI.
Assets 2
Loading