Aggregate results from multiple test runs and generate Allure reports.
Installation steps:
# create python virtual env
python3 -m venv .env
# activate python virtual env
. .env/bin/activate
# update python virtual env
python3 -m pip install --upgrade pip
python3 -m pip install --upgrade wheel
python3 -m pip install --upgrade virtualenv
virtualenv --upgrade-embed-wheels
# install package
python3 -m pip install --upgrade --upgrade-strategy eager -e .
Copy files from example cron job from examples directory and edit them as needed. Namely it is needed to provide the GITHUB_TOKEN (or BUILDKITE_TOKEN) variable.
For nightly results, see update_reports.sh.
Example for publishing results of on-demand testing job on Github:
# fetch results for testrun id '1.35.6rc2-default_mixed_01' (can be multiple jobs)
report-aggregator testrun -d results/testruns -n 1.35.6rc2-default_mixed_01
# aggregate resuls with the same testrun id and publish them
report-aggregator publish --results-dir results/testruns --web-dir /var/www/reports --aggregate