CI Guide

We use the R package tic to specify actions that should be executed on Travis CI.

tic

Why add another layer on top of Travis?

tic is ci-agnostic, meaning that the same script will also be valid on other CI systems (e.g. Appveyor for Windows).
tic makes the deployment of docs easy and transparent.
tic uses R functions instead of Travis own functions.
tic uses rcmdcheck::rcmdcheck() instead of R CMD CHECK. This enhanced version comes with the major among smaller advantages with the feature of showing the full log of failed tests.

In .travis.yml most calls just refer to specfic tic calls. These tic commands are then specified in tic.R. The organization of tic.R is similar to Travis (and CIs in general): It uses "stages", i.e. different levels of logical layers that are executed in a specific order (e.g. before_install runs before stage install).

tic.R is subdivided into groups by environment variables referring to specific build stages.

mlr3

The .travis.yml file should stay static. Everything is redirected to tic.R which lives in mlr-ci and is shared across many repos via a git submodule approach. Since the default cloning approach done by Travis does not automatically use the latest state of the submodule repo, we need to disable the default cloning of Travis. Instead we use the following command manually in the before_install stage of .travis.yml:

git submodule update --init --recursive --remote

The mlr3 .travis.yml file can be used as a template for other repos. Unfortunately, the .travis.yml file cannot be shared via a submodule approach and needs to be created for each repository.

The pkgdown site is deployed into the gh-pages branch. This is done via the tic macro do_pkgdown(). Option orphan = TRUE means that the whole branch is wiped, storing only the most recent deployment.

R CMD Check is run by macro do_package_checks() . The build should fail for errors and warnings, not for notes.

tic.R

do_package_checks(error_on = "warning")
    
if (ci_has_env("BUILD_PKGDOWN")) {
  do_pkgdown(orphan = TRUE)
}

Some packages of the mlr3 package family need some additional system libraries or other modifications of tic.R and .travis.yml. These repos therefore do not share the tic.R file mentioned above from the mlr-ci repo but have their own distinct file.

For some packages containing compiled code you need to add argument document = FALSE to the do_pkgdown() macro. This has to do with an issue going back to package pkgbuild that has some issues with compiled code in packages. See tic.R in mlr3survival for an example. The mentioned error looks as follows:

Error in getNativeSymbolInfo(symNames, dll, unlist = FALSE, withRegistrationInfo = TRUE) : 
  must pass a package name, â€œDLLInfoâ€� or â€œDllInfoReferenceâ€� object

mlr3book

A run of the build is triggered on each run of a mlr3* repo.

This happens via https://github.com/plume-lib/trigger-travis.

The following needs to be put into tic.R :

get_stage("after_success") %>%
  add_code_step(system('sh inst/trigger-mlr3book.sh'))

Additionally, place the following script in inst/trigger-mlr3book.sh :

echo "TRAVIS_BRANCH=$TRAVIS_BRANCH TRAVIS_PULL_REQUEST=$TRAVIS_PULL_REQUEST"
if [[ ($TRAVIS_BRANCH == master) &&
      ($TRAVIS_PULL_REQUEST == false) ]] ; then
  curl -LO --retry 3 https://raw.github.com/mernst/plume-lib/master/bin/trigger-travis.sh
  sh trigger-travis.sh mlr-org mlr3book $TRAVIS_ACCESS_TOKEN "trigger from $TRAVIS_REPO_SLUG $TRAVIS_COMMIT"
else
  echo "Not triggering a mlr3book deployment"
fi

The TRAVIS_ACCESS_TOKEN of the repo can be queried from the terminal if the travis library is installed locally with travis token from the repo root. If you have it, store as an env variable in the TravisCI repo setting.

mlr

Stages

The first two stages build the R package cache. If the cache exists, these stages will finish quickly as they do nothing more than downloading the cache. If the cache needs to be rebuild, stage 1 will install all packages listed in the environment variable WARMUPPKGS. Stage 2 then installs all dependencies of mlr. To avoid timing out in a stage, WARMUPPKGS were selected in that way that the runtime is somewhat equally distributed across both warmup stages. Another word to caching: Cache is only shared between stages with the same environment variable at the time of writing this. However, there is already an open PR so that the cache can be shared across build stages with different env variables: https://github.com/travis-ci/travis-build/pull/1265. This is important as we currently need to build two caches: For the package checking and tutorial creation.
Stage check makes sure that all dependencies are installed and up-to-date and performs the checking of the package including all tests
Stage 4 builds the HTML version of the tutorial
Stage 5 builds the PDF version of the tutorial

Tutorial

The tutorial is build using the most recent version of pkgdown and deployed to the gh-pages branch. This way we do not clutter the main repo (an alternative would be to deploy to the docs/ directory of the master branch).
The deployment is done via an SSH key stored in the Travis repo setting. The key was added using travis::use_travis_deploy(). Deployment is only performed for non-PR commits to the master branch and non-cron builds (cron = automatic daily triggered builds by Travis if no commit happened).

Inserting figures

Differs for the HTML and PDF build due to visual appearance of the rendered graphics. You need to specify an HTML and PDF chunk within the tutorial page. Execution is then conditioned on the respective rendering type. Example:

knitr::include_graphics("img/nested_resampling.png")

knitr::include_graphics("<https://raw.githubusercontent.com/pat-s/mlr/master/vignettes/tutorial/devel/pdf/img/nested_resampling.png>")

For some reason we need to use the raw Github URL path for the HTML version as rendering will not work otherwise. Maybe this was fixed meanwhile (I do not know what caused the error in the first place) so you can give it a try using relative paths similar as being used for the PDF version.

TODO:

mention pkgdown.yml and templates

Travis debug mode

Get your API token from your profile page at: https://travis-ci.org/profile
Send a POST request to /job/:job_id/debug replacing the TOKEN and JOB_ID values below:

#! /usr/bin/env bash
curl -s -X POST \ -H "Content-Type: application/json" \ -H "Accept: application/json" \ -H "Travis-API-Version: 3" \ -H "Authorization: token " \ -d '{ "quiet": true }' \ https://api.travis-ci.org/job/<JOB_ID>/debug

The Job ID is displayed in the build log after expanding "Build system information".

Head back to the web UI and in the log of your job. There you should see the following lines to connect to the VM:

 Setting up debug tools.
 Preparing debug sessions.
 Use the following SSH command to access the interactive debugging environment:
 ssh [[email protected]](mailto:[email protected])