Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"CI" tag for system dependencies only #312

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,12 @@ image | description
[tidyverse](https://hub.docker.com/r/rocker/tidyverse) | Adds tidyverse & devtools | [![](https://img.shields.io/docker/pulls/rocker/tidyverse.svg)](https://hub.docker.com/r/rocker/tidyverse)
[verse](https://hub.docker.com/r/rocker/verse) | Adds tex & publishing-related packages | [![](https://img.shields.io/docker/pulls/rocker/verse.svg)](https://hub.docker.com/r/rocker/verse)
[geospatial](https://hub.docker.com/r/rocker/geospatial) | Adds geospatial packages on top of 'verse' | [![](https://img.shields.io/docker/pulls/rocker/geospatial.svg)](https://hub.docker.com/r/rocker/geospatial)
[ci](https://hub.docker.com/r/rocker/ci) | r-ver with system dependencies for tidyverse, verse, and geospatial, no R packages | [![](https://img.shields.io/docker/pulls/rocker/ci.svg)](https://hub.docker.com/r/rocker/ci)
[shiny](https://hub.docker.com/r/rocker/shiny) | Adds shiny server on top of 'r-ver' | [![](https://img.shields.io/docker/pulls/rocker/shiny.svg)](https://hub.docker.com/r/rocker/shiny)
[shiny-verse](https://hub.docker.com/r/rocker/shiny-verse) | Adds tidyverse packages on top of 'shiny' | [![](https://img.shields.io/docker/pulls/rocker/shiny-verse.svg)](https://hub.docker.com/r/rocker/shiny-verse)
[binder](https://hub.docker.com/r/rocker/binder) | Adds requirements to 'geospatial' to run repositories on [mybinder.org](https://mybinder.org/) | [![](https://img.shields.io/docker/pulls/rocker/binder.svg)](https://hub.docker.com/r/rocker/binder)
[cuda](https://hub.docker.com/r/rocker/cuda) | Adds cuda and Python to 'r-ver' (a.k.a `rocker/r-ver:X.Y.Z-cuda10.1`) | [![](https://img.shields.io/docker/pulls/rocker/cuda.svg)](https://hub.docker.com/r/rocker/cuda)
[ml](https://hub.docker.com/r/rocker/ml) | Adds rstudio server, tiyverse, devtools to 'cuda' | [![](https://img.shields.io/docker/pulls/rocker/ml.svg)](https://hub.docker.com/r/rocker/ml)
[ml](https://hub.docker.com/r/rocker/ml) | Adds rstudio server, tidyverse, devtools to 'cuda' | [![](https://img.shields.io/docker/pulls/rocker/ml.svg)](https://hub.docker.com/r/rocker/ml)
[ml-verse](https://hub.docker.com/r/rocker/ml-verse) | Adds tex & publishing-related packages & geospatial packages to 'ml' | [![](https://img.shields.io/docker/pulls/rocker/ml-verse.svg)](https://hub.docker.com/r/rocker/ml-verse)

**For more information on public container images, please see [the Wiki of this repository](https://github.com/rocker-org/rocker-versioned2/wiki).**
Expand Down
118 changes: 118 additions & 0 deletions scripts/install_renv_sysdeps.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
#!/bin/bash
set -e

## build ARGs
NCPUS=${NCPUS:--1}

# always set this for scripts but don't declare as ENV..
export DEBIAN_FRONTEND=noninteractive
export PATH=$PATH:/usr/local/texlive/bin/x86_64-linux/

apt-get update

#----- SysDeps for Tidyverse-----
apt-get install -y --no-install-recommends \
file \
git \
libapparmor1 \
libgc1c2 \
libclang-dev \
libcurl4-openssl-dev \
libedit2 \
libobjc4 \
libssl-dev \
libpq5 \
lsb-release \
psmisc \
procps \
python-setuptools \
pwgen \
sudo \
wget

#----- SysDeps for Verse-----
apt-get install -y --no-install-recommends \
cmake \
default-jdk \
fonts-roboto \
ghostscript \
hugo \
less \
libbz2-dev \
libglpk-dev \
libgmp3-dev \
libfribidi-dev \
libharfbuzz-dev \
libhunspell-dev \
libicu-dev \
liblzma-dev \
libmagick++-dev \
libopenmpi-dev \
libpcre2-dev \
libv8-dev \
libxml2-dev\
libxslt1-dev \
libzmq3-dev \
qpdf \
texinfo \
software-properties-common \
vim

# librdf0-dev depends on libcurl4-gnutils-dev instead of libcurl4-openssl-dev...
# So: we can build the redland package bindings and then swap back to libcurl-openssl-dev... (ick)
# explicitly install runtime library sub-deps of librdf0-dev so they are not auto-removed.
apt-get install -y librdf0-dev
install2.r --error --skipinstalled -n $NCPUS redland
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cboettig Not sure what is needed in this section

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, this is a bit of a mess. debian/ubuntu ship a few different versions of libcurl (specifically, gnutils vs openssl), which conflict. (Some packages can bind either one, but don't necessarily play nicely with both -- e.g. I believe curl R package doesn't fail but does some unexpected things with the non-openssl version iirc?)

The redland R package needs librdf0-dev which depends on gnutils, meaning that line 64 will uninstalls the openssl version and potentially breaks things. So if we're going to install librdf0-dev, I think it is necessary to reinstall libcurl4-openssl-dev after we finish building redland from source (maybe not necessary now that we can get binaries??)

Anyway, long story short, I've always been very confused about the libcurl conflict. @eddelbuettel or @jeroen probably have a much better handle on this; maybe one of them could advise what's the best way to support both redland and curl R packages?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, do not have a fix. There are alternative implementations, which in turn are use by different packages, so you could indeed end up with 'graph' with conflicts. Best solution might be to locally rebuild packages using, say, gnutls to use openssl. Not something I tried though.

apt-get install -y \
libcurl4-openssl-dev \
libxslt-dev \
librdf0 \
redland-utils \
rasqal-utils \
raptor2-utils \
&& apt-get remove -y systemd \
&& apt-get -y autoremove

apt-get install -y libgit2-dev libcurl4-openssl-dev

## Add LaTeX, rticles and bookdown support
wget "https://travis-bin.yihui.name/texlive-local.deb" \
&& dpkg -i texlive-local.deb \
&& rm texlive-local.deb

## Install texlive and pandoc
/rocker_scripts/install_texlive.sh
/rocker_scripts/install_pandoc.sh

#----- SysDeps for Geospatial-----

apt-get install -y --no-install-recommends \
gdal-bin \
lbzip2 \
libfftw3-dev \
libgdal-dev \
libgeos-dev \
libgsl0-dev \
libgl1-mesa-dev \
libglu1-mesa-dev \
libhdf4-alt-dev \
libhdf5-dev \
libjq-dev \
libpq-dev \
libproj-dev \
libprotobuf-dev \
libnetcdf-dev \
libsqlite3-dev \
libudunits2-dev \
lsb-release \
netcdf-bin \
postgis \
protobuf-compiler \
sqlite3 \
tk-dev \
unixodbc-dev


#----- Cleanup -----
rm -rf /var/lib/apt/lists/*