Builds for different versions of GDAL, PROJ, GEOS

[Robinlovelace]

As discussed in r-spatial/discuss#35 it's useful to be able to control the upstream spatial libraries on which R's geographic infrastructure builds, for reproducibility and development work. I have created an issue in our geocompr book project to support this: geocompx/geocompr#476

To minimise re-inventing the wheel, I wonder if it would be possible to build from version-tags of rocker/geospatial e.g.

FROM rocker/geospatial:proj7 # for proj 7 support
FROM rocker/geospatial:ubuntugis-unstable # for the latest version of the ubuntugis-instable repo
# ...

[cboettig]

Thanks @Robinlovelace , this is definitely something we want to support in rocker-versioned2. For us, I think the key piece is just knowing what versions/combinations would be useful to publish tags for, and what those tags should be. (i.e. in a proj7 tag, what version of GDAL & GEOS does it have? do we need to support the different GEOS versions or just the proj versions? do we still need proj4 support in newer images?

You can get a sense of how the new system will specify a rocker stack along with specific versions of everything like so: https://github.com/rocker-org/rocker-versioned2/blob/master/versions-cuda.json

[Robinlovelace]

I think the key piece is just knowing what versions/combinations would be useful to publish tags for, and what those tags should be. (i.e. in a proj7 tag, what version of GDAL & GEOS does it have?

That is the key question for me too but I don't feel qualified to answer. A starter for 10 could be:

• current GDAL versions 2 and 3 as documented here: https://gdal.org/download.html (remembering that " For all x.y.z versions you should always choose the highest z, because these were bug fixes" )
• ubuntugis and ubuntugis-unstable repos, the latter of which stays fairly close to the bleeding edge and is already shipping PROJ 7: permit PROJ 7 r-spatial/sf#1254 (comment)
• The combinations used in PostGIS's automated build testing system (which I don't fully understand): https://github.com/postgis/postgis/blob/master/.travis.yml

There are many people who are well qualified to make informed suggestions, @edzer being the first person who comes to mind (sorry for the nudge, hope this will be useful for r-spatial).

[Robinlovelace]

You can get a sense of how the new system will specify a rocker stack along with specific versions of everything like so: https://github.com/rocker-org/rocker-versioned2/blob/master/versions-cuda.json

Looking at how you can seemingly use partial docker images, this may also be of use:

https://github.com/OSGeo/gdal/tree/master/gdal/docker

[Robinlovelace]

Quickfire question for @cboettig (having already checked-in with Dirk). Please could you take a scan of this and let me know of anything you'd flag as unhelpful or in need of improvement before it goes out (plan is for Monday afternoon): https://github.com/geocompr/geocompr.github.io/blob/source/content/post/2020/installing-r-spatial-packages-linux.md

Mentions rocker/geospatial and alludes to this very issue. Thank you!

[cboettig]

apologies for not getting to this on time but great blog post!

[Robinlovelace]

Hope it helps with your efforts. Any update on that (I regard any work done in the current context as a bonus ; ) ?

[cboettig]

it's in my queue! remote teaching my students and homeschooling my kids has taken a toll on my rocker development!

[Robinlovelace]

Heads-up @cboettig, another variation in build matrices should be RStudio set-up options. I have demonstrated the feasibility of this, building on the new shiny rockerdev/geospatial:3.6.3 image and it seems to work: geocompx/docker@d14dc4a

See the readme of that repo for the rather manual way I'm approaching this, inspired by your impressive work in https://github.com/rocker-org/rocker-versioned2

Let me know if I can be of any help, very glad to see stable docker images with many tags building from Ubuntu on the production line!

[cboettig]

@Robinlovelace Okay, finally getting back to this, apologies.

So far we have these flavors:

• rockerdev/geospatial:3.6.3, which builds with only the ubuntu:bionic default repos, providing: GEOS 3.6.2, GDAL 2.2.3, PROJ 4.9.3:

• rockerdev/geospatial:3.6.3-unstable, also on ubuntu:bionic but adds the ubuntugis-unstable PPA, which gives: GEOS 3.8.0, GDAL 3.0.4, PROJ 7.0.0

Unfortunately, I'm hitting a problem on the upcoming release (ubuntu:focal aka 20.04; also same issue on debian:testing). It seems liblwgeom-dev has disappeared... There aren't libs available focal yet in ubuntugis ppa either so that won't help yet. Do you have any leads on that issue? Why would lwgeom-dev have disappeared?

We may end up creating both 4.0.0-bionic and 4.0.0-focal tags in order to support the older distro for a bit (potentially we could also go back to 4.0.0-xenial to support the 16.04 LTS release).

[cboettig]

@Robinlovelace nevermind, looks like we do not need liblwgeom-dev in the images that have newer PROJ libraries.

So building on Ubuntu 20.04 we get:

docker run --rm -ti rockerdev/geospatial:devel Rscript -e "library(sf)"
Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1

So for 4.0.0, I think we will build the following tags for rocker/geospatial:

• 4.0.0 / latest: GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
• 4.0.0-bionic: GEOS 3.6.2, GDAL 2.2.3, PROJ 4.9.3
• 4.0.0-unstable (will have to be bionic-based until we have focal in the PPA, and will have unstable/sliding versions of libraries), currently at: GEOS 3.8.0, GDAL 3.0.4, PROJ 7.0.0

How does this sound?

[Robinlovelace]

That sounds ideal, and will simplify the set-up for https://hub.docker.com/r/geocompr/geocompr

Looking forward to testing geographic packages on R 4.0.0 and, looking at https://hub.docker.com/r/rockerdev/geospatial/tags it seems they're working 🎉

[cboettig]

Debating whether we keep alive the older ubuntu18.04 tags that provide the PROJ 4.9 (in the vanilla repos) and PROJ7 (via the unstable ppa) support?

The builds are fine and all, but mostly trying to avoid confusion with our tagging conventions -- i.e. do need to have tags that explicitly refer to both ubuntu20.04 and ubuntu18.04 then, vs just saying ">= 4.x builds on latest LTS", which at this time would mean everything is 20.04... In particular, https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable now has PPA support for focal, so we could add a bleeding edge unstable tag based on the Ubuntu focal images, instead of having that provided by the 18.04? we would lose support for old Proj 4.x of course, though it would still be available on the rocker 3.x images....

Thoughts?

[Robinlovelace]

so we could add a bleeding edge unstable tag based on the Ubuntu focal images, instead of having that provided by the 18.04?

Sounds like a good plan to me.

[Robinlovelace]

Update on this @cboettig, I see that PROJ 7.0 is currently only available on 18.04 currently. I'm wondering when they will support it in ubuntugis-unstable or even the stable PPA. May be worth asking a question here: https://lists.osgeo.org/mailman/listinfo/ubuntu

I think we need a way to get the latest version of PROJ and having Ubuntu 18.04 with ubuntugis-unstable PPA is a pretty solid (despite the PPA's name) way of making that happen. Has the suggested fix been implemented? I can give it a try if so.

Seems Focal may be a bit close to the cutting edge but I think you guys made the right decision by migrating early.

[eddelbuettel]

Seems Focal may be a bit close to the cutting edge but I think you guys made the right decision by migrating early.

Care to expand? It is a full-blown LTS release with full supported and a first-class release status.

[Robinlovelace]

Sure:

Source: https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable?field.series_filter=focal

Vs

https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable?field.series_filter=bionic

Hopefully soon now that the point release is almost released they will update the UbuntuGIS PPA with full support for Focal which would make this true for geographic libraries (currently the release version of PROJ is 6.3 which has known issues). I think that it will have full support within a month or so, not a moment too soon!

[eddelbuettel]

I see. I missed that the comment was about what ubuntugis has, or has not, offered so far. They may well be behind.

Focal (and when left unqualified meaning the actual distro) is actually fine and stable, at least on my machines but also from what one can read more broadly online. We were just focussing on different sets here.

[cboettig]

Yeah, i still need to prep a focal-based unbuntugis:unstable... on the todo list

On Mon, Jul 27, 2020 at 9:59 AM Dirk Eddelbuettel ***@***.***> wrote: I see. I missed that the comment was about what ubuntugis has, or has not, offered so far. They may well be behind. Focal (and when left unqualified meaning the actual distro) is actually fine and stable. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABWK6WXLHADWJUHRKF4D73R5WW7TANCNFSM4LKWG3GQ> .

-- --- Carl Boettiger http://carlboettiger.info/

[Robinlovelace]

Yeah, i still need to prep a focal-based unbuntugis:unstable... on the todo

May as well wait until there is actually anything (more than 3 packages) in the ubuntugis-unstable repo. I agree with @eddelbuettel that Focal as a distro is solid and that they are behind. Personally I'm waiting for the point release before I consider Focal an LTS and before I upgrade my work computers and I imagine the team may be thinking likewise. One way to elicit what they are thinking is by asking them here (I'm tempted, could ask on Rocker's behalf).

[Robinlovelace]

This raises the parallel question: what is the status of Bionic-based Rocker images, will they be maintained for the next year or so?

[cboettig]

@Robinlovelace sorry I've been mostly away from my computer and didn't read your thread carefully on my phone -- now I see what you mean about the focal version of unstable PPA being pretty empty!

Yeah, I think keeping 18.04 alive is not off the table, we are finding a few other edge cases (some weirdness with CyVerse internals for instance) that also have issues on focal but not bionic. I think my main concern with doing so is the confusion with the tags -- e.g. because we have some ubuntu18.04 tags on images, we've seen a good number of users assume that the new 4.0 images that don't have ubuntu18.04 are still debian based. We could tag everything with ubuntu20.04 as an optional tag, like nvidia does, but might get confusing fast...

I do think it would be a very good idea to make contact with the Ubuntu-gis PPA maintainers and hear there thoughts on the matter. It seems a bit weird that unstable PPA would not package a newer Proj library on focal, and so maybe they are still rolling that out. (on the other hand, the stuff they already have up there needs only proj6 by the look of it).

[Robinlovelace]

I say lets revisit this after 6th August after the point release is out. I'm in favour of contacting the ppa maintainers if there are no big changes by early September and it may be worth reaching out before then.

About tags... If it's good enough for Nvidia it's good enough for Rocker is my thinking.

Many thanks for getting this stuff up and running, wrinkles are inevitable and Ubuntu is solid (although I do wonder in a grass-is-greener way what a Rocker container based on Clear Linux base image would look like - even more time consuming to set-up and maintain I imagine!).

[cboettig]

@Robinlovelace haven't forgotten this thread. Have you had a chance to reach out to the ubuntugis folks? Looks like they link a mailing list https://lists.osgeo.org/mailman/listinfo/ubuntu, on their PPA, I haven't subscribed yet. Looks like they might end up pointing us to the debian-gis group, https://debian-gis-team.pages.debian.net/, from which most of their stuff is derived anyway and may be more active right now?

Apologies I don't have enough bandwidth to reach out those folks now; but I do think it would be preferable to have a the unstable tag be based on the latest ubuntu release instead of maintaining bionic just for that. Alternately, we may want to go back and revive our old recipes for simply installing GDAL, GEOS, and PROJ from source? With the new modular script system, that should be quite nice and would have some advantages over the PPA route?

[Robinlovelace]

Hey @cboettig I did send and email to the list but it bounced. Can try again with the osgeo/debian-gis lists you link to and let you know if I have any more luck, but probably won't be until September... I would be surprised if UbuntuGIS didn't fully support Focal now that it's officially the LTS and imagine they're just waiting to update the PPA, in which case reviving the 18.04 wouldn't be necessary.

[Robinlovelace]

Alternately, we may want to go back and revive our old recipes for simply installing GDAL, GEOS, and PROJ from source? With the new modular script system, that should be quite nice and would have some advantages over the PPA route?

I think that is worth doing anyway and reduces dependencies on other people.

[cboettig]

@Robinlovelace I took a quick look at doing this with https://github.com/rocker-org/rocker-versioned2/blob/master/scripts/install_geospatial_unstable.sh. Doesn't seem to build on verse right now though because GEOS 3.8.0 complains it can't find PROJ 6 files (note the script installs PROJ 7.1.0 from source instead).

GEOS 3.8.0 builds for me if I build this script on top of rocker/geospatial (where PROJ 6 is already available from the ubuntu repos), but this seems to upset sf, which complains that lwgeom is using PROJ 7 while it has PROJ 6.

There's probably a well-known solution to this but might take a bit more futzing around or messaging some mailing lists.

Also, just wanted to flag that regardless of how we get the newer libs installed, it will raise issues with the current default of installing binary R packages, since they will be compiled against the standard ubuntu versions. I'm currently trying to work around that on an ad-hoc basis by installing from source only for those R packages which I know bind these 3 libraries, but maybe might be safer to switch the default to install all R packages from source for the geospatial-unstable image?

Other thoughts:

• not sure if unstable is the right tag if we're building these libraries from source. Technically it means we could build any version requested with this script. unstable might suggest we're using the PPA, which we aren't? But can't think of an equally suggestive moniker.

• one more option would be to add the bionic PPA to the focal-based image (pretty sure this works, at least if you manually add the relevant line to sources.list. Maybe not advisable but pretty sure using older PPAs on a newer distro works, though vice versa usually doesn't).

[Robinlovelace]

Many thanks for delving further into this Carl. The most simple and future-proof suggestion seems to be to add the bionic PPA to image, assuming that works.

Regarding your question

I'm currently trying to work around that on an ad-hoc basis by installing from source only for those R packages which I know bind these 3 libraries, but maybe might be safer to switch the default to install all R packages from source for the geospatial-unstable image?

Yes I think it would be safer to install all packages on the geospatial-ubuntu images from source.

Those are just my first thoughts and I'm a bit out of my depth. Worth asking developers of geographic R packages who use Ubuntu. Apologies for the tag but what are your thoughts on this @edzer? Would a 20.04 image pointing to the 18.04 version of the UbuntuGIS unstable PPA be useful? These supported images, with different versions of OSGeo libraries, could help with testing of sf and other packages against different versions of GDAL and friends.

I imagine the best solution from an R developer's perspective would be to be able to test against the latest versions shipped by OSGeo. That could also be the best solution from a performance perspective. OSGeo has pretty good documentation on building from source I think and has Docker images with a range of configurations for PROJ and GDAL: https://hub.docker.com/u/osgeo

So my inclination would be ambitious in terms of building from source (in addition to supporting the UbuntuGIS PPA somehow) but that's easy for me to say as I'm not volunteering to do the skilled work of getting these images working, although am very happy to provide further ideas/feedback. Important to get strong foundations on which many other things can build.

[Robinlovelace]

Update, I tried adding these lines into /etc/apt/sources.list:

deb http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic main
deb-src http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu bionic main

It updated PROJ which seemed to have worked by sf is still linked to PROJ 6.3.1, possibly due to previous binary installation...

[cboettig]

images are now live, e.g.

docker run --rm -ti rocker/geospatial:4.0.3-ubuntugis R -e "library(sf)"
## identically
docker run --rm -ti rocker/geospatial:ubuntugis R -e "library(sf)"

## older R
docker run --rm -ti rocker/geospatial:4.0.2-ubuntugis R -e "library(sf)"

[Robinlovelace]

Great, plan to take a look tomorrow. Many thanks for amazing work Carl! I'm happy that this is 'good enough' to solve the purposes identified in the original post. However I will keep it open in case there is a possibility of getting dev versions of OSGeo packages installed from source. Looking at r-spatial/sf#1510 I think such a rocker/geospatial:dev-osgeo image could be useful and am happy to help efforts to make that happen.

[cboettig]

Thanks @Robinlovelace . My adaption of the OSGEO recipe is in the /rocker_scripts/install_gdal_source.sh on all the r-ver containers. PROJ_VERSION and GDAL_VERSION can be set in this script as env vars, or default to master like they do in the original OSGEO recipe.

Now I think I did it right, but for reasons I haven't figured out, sf seems not to be able to find some bits:

FROM rocker/verse
## calls install_geos_source,sh and install_proj_source.sh, default to version `master`
RUN /rocker_scripts/install_gdal_source.sh

## install sf from source?
RUN install2.r -r https://cran.r-project.org sf

gives me the error:

checking for gdal-config... /usr/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.2.0
checking GDAL version >= 2.0.1... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /usr/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... yes
configure: GDAL: 3.2.0
configure: pkg-config proj exists, will use it
configure: using proj.h.
configure: PROJ: 7.2.0
checking PROJ: checking whether PROJ and sqlite3 are available for linking:... no
configure: error: libproj or sqlite3 not found in standard or given locations.
ERROR: configuration failed for package ‘sf’
* removing ‘/usr/local/lib/R/site-library/sf’

The downloaded source packages are in
        ‘/tmp/downloaded_packages’
Warning message:
In install.packages(pkgs, ...) :
  installation of package ‘sf’ had non-zero exit status

So it's successfully detecting proj version we have installed, but not finding the libs for proj. (even more confused why it's not finding the libs for sqlite3, since libsqlite3-dev is just installed from apt-get....

maybe @edzer can spot what I've done wrong in linking

[Robinlovelace]

Great reproducible example @cboettig it may be worth opening an issue on the sf tracker on this, should be able to find dev versions of these packages, these 'dev' containers will be very useful for debugging I think. Apologies for more unsolicited tagging but heads-up @rsbivand - this should help efforts to build against the latest versions of OSGeo packages so any suggestions welcome. Thanks.

[Robinlovelace]

Heads-up @cboettig I'm testing this roughly as follows (reproducible example from imperfect memory):

git clone [email protected]:rocker-org/versioned2
cd versioned2
docker run --rm -v $(pwd):/home/rstudio/data --it rocker/verse
bash /home/rstudio/data/scripts/install_gdal_source.sh

It's taking ages, building GDAL from source looks like a monster operation best done upstream in docker image builds so confident this is well suited to dockerisation. Still compiling / doing what it needs to do... Does that approach to debugging this GDAL dev issue seem reasonable to you? Once that is up-and-running and the sf install issue is fixed I think we can finally close this issue, assuming that approach is future-proof and also builds dev versions of GEOS and PROJ.

[Robinlovelace]

Heads-up @cboettig I've put in a PR that should fix the issue: rocker-org/rocker-versioned2#92

[edzer]

Looks very similar to r-spatial/sf#1268 (comment) where the problem seemed related to proj, not libsqlite3-dev.

[Robinlovelace]

Hi @edzer thanks for the comment + link but I think I've fixed it in the PR above.

[Robinlovelace]

Will follow-up in the other issues though to join things up, great someone's on the same track!

[Robinlovelace]

Any luck getting this image built @cboettig ? I think it works locally... Disclosure of interest: want to use it as the basis of new Dockerfiles in the geocompr/docker repo.

[cboettig]

@Robinlovelace yup, I think it's working. just waiting for the main build to re-construct the stacks now (since all the rocker_scripts/ are made available in all images, my make rebuilds all the stacks for all the tags which takes ~ 12+ hrs)

[Robinlovelace]

Heads-up @cboettig I just took a look here and cannot see any 'osgeo-dev' or similar tagged images: https://hub.docker.com/r/rocker/geospatial/tags

[cboettig]

yeah apologies, still not up yet. students got a bit ambitious yesterday and crashed our server that does the builds... hopefully up soon, I'll ping

[Robinlovelace]

Good to hear of ambitious students! Good on them for pushing the boundaries. Keep me posted, sorry for the nudging : )

[cboettig]

@Robinlovelace actually looks like I'm still having the same issue in being unable to install sf due to not finding sqlite3, even with the merged change. Same error as before, looks like r-spatial/sf#1518 (comment) is seeing the same thing I am.

[ranghetti]

Hi @cboettig and @ranghetti now I'm seeing a different issue. I get:

cd /tmp
git clone [email protected]:rocker-org/rocker-versioned2
cd rocker-versioned2
docker run --rm -ti -v $(pwd):/home/rstudio/data rocker/verse /bin/bash
bash /home/rstudio/data/scripts/install_gdal_source.sh
install.packages("sf", repos = "https://cloud.r-project.org")
* installing *source* package ‘sf’ ...
** package ‘sf’ successfully unpacked and MD5 sums checked
** using staged installation
configure: CC: gcc
configure: CXX: g++ -std=gnu++11
checking for gdal-config... no
no
configure: error: gdal-config not found or not executable.
ERROR: configuration failed for package ‘sf’
* removing ‘/usr/local/lib/R/site-library/sf’
* restoring previous ‘/usr/local/lib/R/site-library/sf’

The downloaded source packages are in
        ‘/tmp/Rtmp96aywq/downloaded_packages’

Originally posted by @Robinlovelace in r-spatial/sf#1518 (comment)

Hi @Robinlovelace, I continue here the discussion as suggested.

Probably this was due to an error occurred running install_gdal_source.sh, so interrupting it before running lines 190-193. For instance, in my case I had to remove line 167, and I received an error executing projsync --system-directory --all (not all the packages were downloaded -- I skipped that). Moreover, I updated the code chunk which I run in r-spatial/sf#1518 (comment) (I had written two lines in the wrong position).

[Robinlovelace]

Thanks @ranghetti I'm trying this again in the latest version of rocker-versioned2 to try to identify what is failing:

docker run --rm -ti -v $(pwd):/home/rstudio/data rocker/verse /bin/bash
bash /home/rstudio/data/scripts/install_gdal_source.sh

[Robinlovelace]

Heads-up @ranghetti, inspired by your work, I've also created a placeholder that builds on the osgeo/gdal:ubuntu-small-latest image. Not wanting to re-invent the wheel I've only put 2 lines in that Dockerfile for now (thinking of using the build scripts in the versioned2 repo, seem reasonable @cboettig - or could try doing it in that repo but don't fully understand the more sophisticated build system there): https://github.com/geocompr/docker/pull/7/files

[Robinlovelace]

Heads-up @cboettig this looks relevant: r-spatial/sf#1268 (comment)

[ranghetti]

Unfortunately in my case this did not help:

cd /tmp
git clone [email protected]:rocker-org/rocker-versioned2
cd rocker-versioned2
docker run --rm -ti -v $(pwd):/home/rstudio/data rocker/verse /bin/bash
rm -R /rocker_scripts/
ln -s /home/rstudio/data/scripts/ /rocker_scripts
export PKG_CONFIG_PATH=/path/to/proj/lib/pkgconfig/
bash /home/rstudio/data/scripts/install_gdal_source.sh
Rscript -e 'remotes::install_github("r-spatial/sf", configure.args=("--with-proj-lib=/usr/local/lib --with-proj-include=/usr/local/include --with-proj-share=/usr/local/share/proj --with-proj-data=/usr/local/share/proj"))'

An error interrupted the installation before reaching step checking PROJ: checking whether PROJ and sqlite3 are available for linking:...:

Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘sf’ ...
** using staged installation
configure: CC: gcc
configure: CXX: g++ -std=gnu++11
checking for gdal-config... /usr/bin/gdal-config
checking gdal-config usability... yes
configure: GDAL: 3.2.0
checking GDAL version >= 2.0.1... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking gdal.h usability... yes
checking gdal.h presence... yes
checking for gdal.h... yes
checking GDAL: linking with --libs only... yes
checking GDAL: /usr/share/gdal/pcs.csv readable... no
checking GDAL: checking whether PROJ is available for linking:... yes
checking GDAL: checking whether PROJ is available fur running:... double free or corruption (out)
./configure: line 3583: 43855 Aborted                 (core dumped) ./gdal_proj
no
configure: error: OGRCoordinateTransformation() does not return a coord.trans: PROJ not available?
ERROR: configuration failed for package ‘sf’
* removing ‘/usr/local/lib/R/site-library/sf’
Error: Failed to install 'sf' from GitHub:
  (converted from warning) installation of package ‘/tmp/Rtmpzk76Vv/filea205969a926/sf_0.9-7.tar.gz’ had non-zero exit status
Execution halted

I will deepen that as soon as I will have time (the step checking whether PROJ is available fur running had not caused an error during my previous try, last month).

[cboettig]

@Robinlovelace and co:

Using @edzer 's Dockerfile as a base, I believe we now have a working dev_osgeos.sh install script which is used to generate the rocker/geospatial:dev-osgeo image (as defined by geospatial-dev-osgeo.json stack file). Note that this is currently also pulling the GitHub versions of rgdal, lwgeom, sf, stars, etc as in Edzer's recipe. Witness:

docker run --rm -ti rocker/geospatial:dev-osgeo R -e "library(sf)"

R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"
> library(sf)
Linking to GEOS 3.8.1, GDAL 3.1.1, PROJ 7.1.0

Importantly, this image should allow GEOS_VERSION, GDAL_VERSION, and PROJ_VERSION to be set as environmental variables. because /rocker_scripts/dev-osgeos.sh can be found on any image, this should mean that it's not too hard for any downstream container building on any rocker image to just set the desired version env vars and run the script (and wait several hours to compile...) and have any gdal version. Possibly the defaults should be bumped or set to 'latest' by default.

Compare this to latest tag: where we see

docker run --rm -ti rocker/geospatial R -e "library(sf)"

R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"
> library(sf)
Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1

and also ubuntugis PPA-based tag, where we see:

docker run --rm -ti rocker/geospatial:ubuntugis R -e "library(sf)"

R version 4.0.3 (2020-10-10) -- "Bunny-Wunnies Freak Out"

> library(sf)
Linking to GEOS 3.8.1, GDAL 3.1.3, PROJ 7.1.1

[Robinlovelace]

Great work @cboettig. Shouldn't the dev-osgeo tag be pulling GDAL3.2.* though? That seems to be the latest release: https://github.com/OSGeo/gdal/releases/

[Robinlovelace]

rocker-org/rocker-versioned2#97

[cboettig]

Good call. fixed, now dev-osgeo tag has

Linking to GEOS 3.8.1, GDAL 3.2.0, PROJ 7.2.0

and thus provides the latest current releases now I think, more recent than ubuntugis

and now versions are set explicitly in the stackfile.

Good to close this out?

[Robinlovelace]

👍 great work, epic issue!