Unit Testing Practices

The unit tests in Verigraph are based on Hspec, QuickCheck and HUnit.

Tools

Hspec is used to organize tests into suites, as well as providing some operators to write expectations (e.g. shouldBe, shouldSatisfy and shouldReturn). It also does automatic discovery of test suites. Thus, when creating tests for a new module Foo.hs, you only need to create a single file FooSpec.hs defining its test suite.

QuickCheck is used to pseudo-randomly generate input data for test cases. Its manual provides more information.

For most unit tests, the functions provided by Hspec should suffice. Sometimes, however, richer assertions might be needed. For that, you may use assertions provided by HUnit. Just never use the crazy operators, always use the functions with descriptive names.

Running Tests

The verigraph.cabal file defines a test suite called HSpecTests that runs all discovered test suites. You can run all tests with the following command:

stack test --test-arguments '+RTS -N -RTS'

This might take too long during development, even if it runs tests in parallel. So there are a few strategies for running only some of the tests.

You may want to select only some test suites (or cases) to run. For that, you may use the options --skip PATTERN and --match PATTERN or -m PATTERN. So the call stack test --test-arguments="-m Graph.Morphism" will only run the suites for graph morphisms, while the call stack test --test-argument-"--skip Graph.Morphism" will skip these suites.

Another useful feature is keeping track of failed tests and only re-running those that failed, which is possible with stack test --test-arguments="--failure-report .hspec-failures --rerun --rerun-all-on-success". We have a configuration file in the project root to always have this behaviour.