How To: Unit Testing

The following steps will ensure that code can be thoroughly tested independent of the hardware it will be running on.

Adding Files to the Code Base

1. Add source files that do not depend on any board-specific hardware to Common/component/. For example, Common/component/NewModule/NewInterface.cpp
2. Write a test driver file using the GoogleTest/GoogleMock framework add it under Common/test. This file name will be the name of the module or class followed by "_test", e.g. "NewModule_test.cpp" and "NewInterface_test.cpp". Sometimes it is easier to test class-by-class, while other times it is only necessary to test the module. This depends on the complexity of the module and it is left up to the programmer's discretion to do what they think is clear and easy to maintain
3. With RobotTest open in Eclipse, navigate to the new source and test files within the linked resources "Common" and uncheck "Exclude resource from build" under Properties->C/C++ Build->Settings for all relevant configurations
4. Run the unit tests under all configurations (e.g. Test_F4 and Test_F7) to ensure the new code is portable to all supported hardware options

Unit Testing

Introduction: TDD

Test-driven development (TDD) is a software development methodology which essentially mandates that you write tests before implementing features. The idea here is that you translate your design requirements into separate tests, and then for each test, you implement only the minimum functionality that is required to make it pass. This minimum implementation is referred to as a unit of code; it implements a tightly-scoped feature that is minimally dependent on other code units (well, ideally). This helps keep the code base concise, and it grows it will have 100% test coverage. This is great because if you ever need to make a change, you will always be able to check you did not break the desired behavior since you have the test as a reference.

In this team, we view TDD as a nice reference methodology, but it is not mandated due to the belief that people tend to work best when given freedom to use their own flow, subject to some guidelines. However, there is the expectation that new code will have reasonable test coverage, and mostly importantly that the code is testable. Our team writes unit tests in C++ and uses Google's Test and Mock frameworks. They're open source and they have great documentation & examples, so it's definitely recommended to check those out.

How is it Different for Embedded Systems?

Embedded systems software is intimately involved with the hardware it runs on top of, so any hardware-specific behavior has to either be (1) simulated/emulated, or (2) blackboxed. Blackboxing means you run your code up until the hardware call, and then you provide your own implementation for what happens after that. For example, you could continue on as though that call never happened (such as if you just toggled a GPIO pin), or you can pretend you received a bunch of bytes via DMA over UART. Obviously, such tests do catch issues such as silicon bugs, misuse of hardware APIs, or timing/thread safety, but they do allow you to verify that the algorithmic portion of your code works properly. In summary, blackboxing allows you to test your software components from the comfort of your PC, without needing access to the hardware, which paves the way for great test coverage and rapid development time for your team. Thankfully, blackboxing is easy to do using C++ and Google Mock (although there can be a fair amount of boilerplate!). Please see the hardware mocking tutorial for detailed instructions on how to mock your hardware-dependent code.

Your First Test

First, we need to include the framework headers.

#include <gtest/gtest.h>
#include <gmock/gmock.h>

The TEST() macro is used to create tests. It takes 2 arguments like so:

TEST(ComponentName, TestName){
...
}

The first argument is the name of the software component being tested. Chances are, all tests in the same file should have the same first argument, since each test file should test only 1 software component. The second argument is a description of what should happen in the test. Consider the fictitious example where ComponentName = Queue and TestName = ShouldBeginEmpty. Then "ComponentName TestName" more or less reads as "queue should begin empty", and we understand that the test is going to assert a new queue has no items in it. Assuming the fictitious Queue class in this example provides an isEmpty method, we could implement this test as follows:

TEST(Queue, ShouldBeginEmpty){
    testQ Queue();
    ASSERT_TRUE(testQ.isEmpty());
}

You will notice we created this queue on the stack (i.e. we did not write testQ = new Queue();). This is because in an embedded environment, heap usage (new, malloc) is disallowed due to its non-deterministic behaviour. For example, heap fragmentation could cause an allocation request to fail, which is difficult to handle gracefully. In any case, as a student team we'd like to try to follow industry practices as much as possible, so this rule sticks—even in tests!

Test Harnesses

TODO

Talking With Hardware

TODO: how to fake calls to the hardware to test data processing algorithms