Fuzzing with AFL

Requirements

Let's get this out of the way first.

OS

JQF's integration with AFL has been tested on MacOS and Ubuntu, though it work on any other linux-based system too. If you have a unix-based system where JQF-AFL does not work as intended, please raise an issue.

Installing AFL

Before using JQF to fuzz your Java programs with AFL, makes sure you have afl-fuzz installed. You can either do this in several ways:

• Download the latest tarball from the author's website.
  • Run make in that directory
• Clone the unofficial GitHub repo
  • Run make in that directory
• Install AFL via standard repositories
  • apt-get install afl on Ubuntu/Debian (check version; see below)
  • brew install afl-fuzz on MacOS with Homebrew

JQF requires at least AFL version 2.31b to work, though something more recent is preferred (JQF has been tested with AFL 2.51b at the time of the writing).

Setting AFL_DIR

JQF must be able to find AFL! If you installed afl-fuzz via standard repositories you may skip this step. JQF will either:

• look for the program afl-fuzz in the current PATH (type afl-fuzz in the command-line to see if it works; also check the minimum version as mentioned above), or
• look for the program afl-fuzz in the directory specified by the environment variable AFL_DIR.

The most common way to get started with is to type in bash:

git clone https://github.com/mcarpenter/afl && (cd afl && make)
export AFL_DIR=$(pwd)/afl

git clone https://github.com/rohanpadhye/jqf && (cd jqf && ./scripts/setup.sh)
jqf/bin/jqf-afl-fuzz

Running jqf-afl-fuzz

The program jqf-afl-fuzz is a wrapper around the program afl-fuzz that comes with AFL. The wrapper allows you to specify the classpath of your Java tests and then just the test class and method instead of having to invoke the JVM yourself. You can always pass additional options to the JVM (e.g. system properties) using the environment variable JVM_OPTIONS.

Usage: ./bin/jqf-afl-fuzz [options] TEST_CLASS TEST_METHOD
Options: 
  -c JAVA_CLASSPATH  Classpath used to find your test classes (default is '.')
  -i AFL_INPUT_DIR   Seed inputs for AFL (default is a few seeds of random data)
  -o AFL_OUTPUT_DIR  Where AFL should save fuzz results (default is './fuzz-results')
  -x AFL_DICT        Provide a dictionary to AFL (default is no dictionary)
  -T AFL_TITLE       Customize title banner (default is TEST_CLASS#TEST_METHOD)
  -m MEM_LIMIT       Set a memory limit in MB (default is 8192)
  -d                 Run AFL in fidgety mode (i.e. skip deterministic mutations)

The wrapper program takes arguments similar to afl-fuzz, for specifying the seed directory, results directory and other relevant AFL options.

Let's assume you have written a standalone JQF test (class MyTest with test method fuzzThis) in a directory called tests:

mkdir tests && vi tests/MyTest.java  # See article on 'Writing a JQF test'
javac -sourcepath tests -cp $(jqf/scripts/classpath.sh) tests/MyTest.java

In this example, we use the handy script jqf/scripts/classpath.sh that expands to the classpath containing the JQF classes and its dependencies, so that your test compiles. In most situations, however, you would want to create tests inside your maven or gradle projects and simply add a dependency on JQF in your pom.xml or build.gradle. See Writing a JQF test for more details.

Finally, let's run jqf-afl-fuzz on our test class:

jqf/bin/jqf-afl-fuzz -c tests MyTest fuzzthis
> Performing pilot run...

The wrapper first performs a pilot run without AFL to ensure that you have set-up your classpath correctly, that JQF can load your test class, and that the test method is a public void method with @Fuzz annotations, and that junit-quickcheck can find generators for your arguments. If there was any problem, you should see an error message here. Otherwise, if all goes well, you'll see:

> Pilot run success! Launching AFL now...

And then the AFL status screen as it fuzzes your program. Good luck!

Analyzing crashes (and other fuzzed inputs)

TODO: Talk about how to replay an input using jqf-repro and Fuzz(repro=).

Troubleshooting

TODO: Talk about common issues with not seeing any coverage and using scripts/janala.conf or jqf-repro -i to debug instrumentation.