This folder contains the Regression tests suite for Falco.
You can find instructions on how to run this test suite on the Falco website here.
This step assumes you already built Falco.
Note that the tests are intended to be run against a release build of Falco, at the moment.
Also, it assumes you prepared falco_traces (see the section below).
Requirements
- Python 3.x
- Virtualenv
- grpcurl
Setup and execution
Using virtualenv
the steps to locally run a specific test suite are the following ones (from this directory):
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
BUILD_DIR="../build" avocado run --mux-yaml falco_tests.yaml --job-results-dir /tmp/job-results -- falco_test.py
deactivate
The name of the specific test suite to run is falco_tests.yaml
in this case. Change it to run others test suites.
In case you want to only execute a specific test case, use the --mux-filter-only
parameter as follows:
BUILD_DIR="../build" avocado run --mux-yaml falco_tests.yaml --job-results-dir /tmp/job-results --mux-filter-only /run/trace_files/program_output -- falco_test.py
To obtain the path of all the available variants for a given test suite, execute:
avocado variants --mux-yaml falco_tests.yaml
The falco_traces.yaml
test suite gets generated through the falco_traces.yaml.in
file and some fixtures (scap
files) downloaded from the web at execution time.
-
Ensure you have
unzip
andxargs
utilities -
Prepare the test suite with the following command:
bash run_regression_tests.sh -p -v
The falco_tests_package.yaml
test suite requires some additional setup steps to be successfully run on your local machine.
In particular, it requires some runners (ie., docker images) to be already built and present into your local machine.
-
Ensure you have
docker
up and running -
Ensure you build Falco (with bundled deps)
The recommended way of doing it by running the
falcosecurity/falco-builder
docker image from the project root:docker run -v $PWD/..:/source -v $PWD/mybuild:/build falcosecurity/falco-builder cmake docker run -v $PWD/..:/source -v $PWD/mybuild:/build falcosecurity/falco-builder falco
-
Ensure you build the Falco packages from the Falco above:
docker run -v $PWD/..:/source -v $PWD/mybuild:/build falcosecurity/falco-builder package
-
Ensure you build the runners:
FALCO_VERSION=$(./mybuild/release/userspace/falco/falco --version | head -n 1 | cut -d' ' -f3 | tr -d '\r') mkdir -p /tmp/runners-rootfs cp -R ./test/rules /tmp/runners-rootfs cp -R ./test/trace_files /tmp/runners-rootfs cp ./mybuild/release/falco-${FALCO_VERSION}-x86_64.{deb,rpm,tar.gz} /tmp/runners-rootfs docker build -f docker/tester/root/runners/deb.Dockerfile --build-arg FALCO_VERSION=${FALCO_VERSION} -t falcosecurity/falco:test-deb /tmp/runners-rootfs docker build -f docker/tester/root/runners/rpm.Dockerfile --build-arg FALCO_VERSION=${FALCO_VERSION} -t falcosecurity/falco:test-rpm /tmp/runners-rootfs docker build -f docker/tester/root/runners/tar.gz.Dockerfile --build-arg FALCO_VERSION=${FALCO_VERSION} -t falcosecurity/falco:test-tar.gz /tmp/runners-rootfs
-
Run the
falco_tests_package.yaml
test suite from thetest
directorycd test BUILD_DIR="../mybuild" avocado run --mux-yaml falco_tests_package.yaml --job-results-dir /tmp/job-results -- falco_test.py
In case you want to run all the test suites at once, you can directly use the run_regression_tests.sh
runner script.
cd test
./run_regression_tests.sh -v -d ../build
Just make sure you followed all the previous setup steps.