Feasibility of packaging noether (all of it) in a Docker image

[gavanderhoorn]

The set of Docker images hosted at ros-industrial/docker includes one for Noether (here).

It currently doesn't really build and uses some forks that are no longer maintained or exist.

Seeing as Noether appears to depend on specific versions of VTK and PCL that may not be very easily distributed on different OS, packaging everything up in a Docker image seems like a good way to facilitate using Noether.

Before spending time on this, I was wondering if some of the maintainers/authors here could provide some insight into how feasible this would be. Not necessarily the packaging itself -- as that would seem to be not too difficult -- but actual use of such a packaged Noether distribution.

I haven't checked (all of) the code sufficiently yet to understand whether any nodelets are being used fi that need to be loaded into camera drivers (which would be complicated by the isolation of a Docker container).

I'm also not entirely sure what the main level of abstraction is at which Noether is normally used, or at which level users are supposed to interact with it. Is the ROS API sufficient to get things to work?

In short: if one were to package Noether including all its (immediate) runtime dependencies in a Docker image, could that image be successfully used in an application, completely removing the need to include Noether in the build of that application?

[gavanderhoorn]

Ping?

[gavanderhoorn]

Took a look at the existing Docker image, which was last built 3 months ago (@Levi-Armstrong ?).

This appears to build a version of PCL (AustinDeric/pcl@b4c8652) which is currently around 1860 commits behind PCL master.

It also tracks VTK's default branch, which is a moving target.

In the interest of reproducable builds, would we not want to pin these versions to specific hashes or tags?

It's also interesting that the two versions of the noether image that are available differ in size by almost 1.3 GB. That is quite a difference.

[jrgnicho]

@gavanderhoorn
The latest docker container should be using pcl 1.9.1 and VTK 8.2. If I recall correctly, they were built from static versions (no git tracking) and installed in the usr/loca/ directory so each new build uses the same versions of pcl and vtk. The increase in memory footprint is probably due to the build artifacts and binaries that resulted from building and installing those libraries. The older version of the docker container isn't being used as far as I recall. My original intention was to replace that old one image with the latest but since I don't quite know my way around docker I ended up creating a new image instead.

If it were to be packaged and provided as a docker container I think most of the functionality could be exposed by the ROS interface, (off the top of my head rastering , mesh and pcl cloud filtering). From my experience, we use parts of it as a library and other parts through ROS in the same application.

What I've been doing to facilitate getting the prerequisites in place is to create debians for VTK and PCL and installing them prior to using noether.

[gavanderhoorn]

So, which specific Dockerfile is used for the image that you are referring to? Is it one of these?

The one in ros-industrial/docker does not use PCL 1.9, I'm pretty sure of that. The manually updated image does contain PCL 1.9; the build script in ros-industrial/docker does was not used for that one (or at least, only in an older layer).

[jrgnicho]

It's using the kinetic one, I'm not really sure why the other one "latest" shows that it was last updated 3 months ago

[gavanderhoorn]

Did you build the image manually, on your own PC? Or did you use a Dockerfile?

rosindustrial/noether:kinetic appears to have gained ~2GB approx 2 months ago in a /bin/bash session.

[jrgnicho]

The first, manually in my own PC

[gavanderhoorn]

Ah, ok. That would explain the confusion.

It would probably be good to have a proper Dockerfile for Noether, as it would help with local rebuilds.

[jrgnicho]

I'm ok with that, where would that be hosted and can it just be created from the image?

[gavanderhoorn]

Theoretically: yes. But only if a Dockerfile was used to create it. Seeing as you updated the latest version manually, the last layer only contains /bin/bash as command, not the actual commands you ran.

I'll take a look at writing a proper Dockerfile.

[dave992]

To run noether I was looking to run it from a Docker image as well due to the library requirements and Ubuntu version. I tried using the Docker Hub Noether Image, but the most recent versions are not working, as they miss the PCL library.

In my search, on why the image is not working I came across this issue. My colleague @IKapitaniuk altered the files from ros-industrial/docker/noether to build a working docker image locally, which are shown below:

Dockerfile

# rosindustrial/noether:kinetic
FROM rosindustrial/core:kinetic
LABEL maintainer "[email protected]"
COPY setup_script.bash /
RUN apt-get update && \
    apt-get install -y wget
RUN bash -c "chmod +x /setup_script.bash && /setup_script.bash"
RUN apt-get clean && \
    rm -rf /var/lib/apt/lists/*

setup_script.bash

#!/bin/bash
wget https://github.com/Kitware/VTK/archive/v7.1.1.tar.gz
tar -xzf v7.1.1.tar.gz
mkdir /vtk_build
cd /vtk_build
cmake -DCMAKE_BUILD_TYPE=MinSizeRel ../VTK-7.1.1
make -j$(nproc)
make install
cd ..
wget https://github.com/PointCloudLibrary/pcl/archive/pcl-1.9.1.tar.gz
tar -xzf pcl-1.9.1.tar.gz
mkdir /pcl_build
cd /pcl_build
cmake -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_surface_on_nurbs:BOOL=1 ../pcl-pcl-1.9.1
make -j$(nproc)
make install
rm -rf /VTK-* /pcl* /vtk_build /v7.1.1*

Any input on this? As otherwise, we can make a PR on ros-industrial/docker to update the files.

[gavanderhoorn]

The Dockerfile not working right now is a known issue and I believe @jrgnicho ran into this with @Levi-Armstrong in some other project as well.

I seem to remember @Levi-Armstrong also has a PPA which has .debs available for the required versions of PCL and VTK: ~ros-industrial/+archive/ubuntu/ppa.

[Levi-Armstrong]

I seem to remember @Levi-Armstrong also has a PPA which has .debs available for the required versions of PCL and VTK: ~ros-industrial/+archive/ubuntu/ppa.

Yea, I am still working on PCL, but the vtk one is good.

[dave992]

The Dockerfile not working right now is a known issue and I believe @jrgnicho ran into this with @Levi-Armstrong in some other project as well.

Thanks, that is good to know.

I seem to remember @Levi-Armstrong also has a PPA which has .debs available for the required versions of PCL and VTK: ~ros-industrial/+archive/ubuntu/ppa.

Yea, I am still working on PCL, but the VTK one is good.

Awesome, I will use the VTK .deb then. I do also see a PCL .deb on the PPA, you do not recommend to use that one then?

[Levi-Armstrong]

The PCL is not ready. There is an issue with how it installs.