guix-cwl

A reproducible from source build of cwltool.

This repository is a GNU Guix channel for the Common Workflow Language which can be installed on any Linux distribution (Guix is distribution agnostic). It also provide a ready made Docker container so you can run without GNU Guix.

This is the runtime graph including a recent version of Python. The figure was generated with

guix graph --type=references cwltool |dot -Tsvg > cwltool-references.svg

This package is used in the reproducible WORKFLOW example.

Table of Contents

• Install cwltool
  • With Docker
  • With GNU Guix channel
  • With GUIX_PACKAGE_PATH
• License
• Notice

Install cwltool

With Docker

Requirements: Docker

GNU Guix can create Docker containers. For examples see here.

The current container was created with

guix pack -f docker -S /usr/bin=/bin -S /etc/profile=/etc/profile cwltool
successfully built /gnu/store/mrwfvb02k9xxzmdfnspxbh4mxa9yg5np-docker-pack.tar.gz.drv
/gnu/store/4al6wg6crp6s3ykkhaz2c5ihrhxm50q3-docker-pack.tar.gz

ls -lh /gnu/store/4al6wg6crp6s3ykkhaz2c5ihrhxm50q3-docker-pack.tar.gz -h
-r--r--r-- 2 root root 198M Jan  1  1970 /gnu/store/4al6wg6crp6s3ykkhaz2c5ihrhxm50q3-docker-pack.tar.gz

The container only includes cwl and its dependencies (see above figure)! You may want to add add binutils bash vim to be able to navigate inside the container with

guix pack -f docker -S /usr/bin=/bin -S /etc/profile=/etc/profile \
   cwltool binutils bash vim

Load the docker image with

docker load --input /gnu/store/4al6wg6crp6s3ykkhaz2c5ihrhxm50q3-docker-pack.tar.gz
docker images
REPOSITORY          TAG                                IMAGE ID            CREATED             SIZE
profile             2g9kac4lj9cbfa5yqn687ds0rmmlfpj1   01c69b34c117        49 years ago        645 MB

You can ignore the time stamp. Run cwltool:

docker run 01c69b34c117 /usr/bin/cwltool

The Docker image is available on Dockerhub as https://cloud.docker.com/repository/docker/pjotrpublic/cwltool-1.0.20181012180214

Note that a bare image is useless because it does not include any of your workflow tooling. For example running a cwltool example will complain about a missing ‘echo’ command:

docker run -v `pwd`:/app -w /app 01c69b34c117 /usr/bin/cwltool tests/echo.cwl --inp "test"
/gnu/store/dxr1rfmnq1wkq3fm97bbznnwarndlvrx-cwltool-1.0.20181012180214/bin/.cwltool-real 1.0
Resolved 'tests/echo.cwl' to 'file:///app/tests/echo.cwl'
[job echo.cwl] /app/50olq6f5$ echo \
    test > /app/50olq6f5/out.txt
'echo' not found: [Errno 2] No such file or directory: 'echo': 'echo'

To run the workflow you’ll need to add the required tools to the Docker image (in this case the binutils package). See the reproducible WORKFLOW for an elaborate example.

With GNU Guix channel

Requirements: Guix >= 0.16 (for the channel feature)

On Ubuntu, Debian, Redhat etc., after installing the GNU Guix 0.16.0 Binary and following the instructions to create a build user etc., create or modify the ~/.config/guix/channels.scm file so it includes

;; Add my personal packages to those Guix provides.
(cons (channel
        (name 'guix-cwl)
        (url "https://github.com/genenetwork/guix-cwl.git"))
      %default-channels)

Then run guix pull as usual and a cwltool package is available:

guix pull

Right after you should be able to see the channel with

Updating channel 'guix-cwl' from Git repository at 'https://github.com/genenetwork/guix-cwl.git'...
Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
Building from these channels:
  guix-cwl  https://github.com/genenetwork/guix-cwl.git ac394cf
  guix      https://git.savannah.gnu.org/git/guix.git   cf51b82

After the pull the package should be visible

guix package -A cwltool

It may take a while to download these packages and some may need to be built. For more ready made binary substitutes you can try instead

guix pull --substitute-urls="https://berlin.guixsd.org http://guix.genenetwork.org https://mirror.hydra.gnu.org"

and install (optionally use --substitute-urls again)

guix package -i cwltool -p ~/opt/cwl

Now to run the tool you need to set the paths etc with

. ~/opt/cwl/etc/profile
cwltool --help

renders

cwltool-1.0.20181012180214

With GUIX_PACKAGE_PATH

There is no magic in channels. I.e., you can opt for the GUIX_PACKAGE_PATH from a checked out GNU Guix repo. E.g.

env GUIX_PACKAGE_PATH=../../gnu/channels/guix-cwl/ ./pre-inst-env guix package -i cwltool -p ~/opt/cwl

License

This channel is published under the GPLv3 in line with other GNU Guix package definitions.

Notice

This guix-cwl channel exists because not all dependencies are ready to go into GNU Guix trunk and both CWL and schema-salad still use the older typing_extensions which obsolete with Python 3.7. We ought to make CWL part of GNU Guix proper in time.

Enjoy,

Pjotr Prins