Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a SOS Dockerfile script and a README #1044

Merged
merged 7 commits into from
Mar 23, 2022
Merged

Added a SOS Dockerfile script and a README #1044

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
32daeec
Added a SOS Dockerfile script and a README
kholland-intel Jan 31, 2022
4bf133f
Addressed PR comments
kholland-intel Feb 10, 2022
70759dc
Added steps in the Dockerfile to build Hydra
kholland-intel Feb 10, 2022
4ebf5b5
Fixed OneAPI/clang issue
kholland-intel Mar 4, 2022
a722f1f
Small edit changes in comments and README
kholland-intel Mar 10, 2022
31d97c1
Added a README note regarding UCX errors with icx
kholland-intel Mar 18, 2022
8d32088
Updated Dockerfile to build libs with make -j
kholland-intel Mar 22, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions scripts/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
FROM ubuntu:20.04

#For OneAPI compilers, the compilers must be added to the builders path
#FROM intel/oneapi-basekit:devel-ubuntu18.04
#ENV PATH=/opt/intel/oneapi/compiler/latest/linux/bin:$PATH

ENV SOS_INSTALL=/home/sos
RUN mkdir /home/sos
ENV DEBIAN_FRONTEND=noninteractive
ENV CC=gcc
ENV CXX=g++
# For OneAPI compilers use clang and clang++
#ENV CC=clang
#ENV CXX=clang++
davidozog marked this conversation as resolved.
Show resolved Hide resolved

RUN apt-get update -y && apt-get install -y \
git \
vim \
build-essential \
wget \
automake \
libtool \
flex \
python \
libhwloc-dev

#Packages added on Travis
RUN apt-get install -y libev-libevent-dev \
libev-dev \
mpich

# Build Libevent
WORKDIR $SOS_INSTALL
RUN wget -c https://github.com/libevent/libevent/releases/download/release-2.1.10-stable/libevent-2.1.10-stable.tar.gz
RUN tar -xzvf libevent-2.1.10-stable.tar.gz
WORKDIR libevent-2.1.10-stable
RUN ./autogen.sh
RUN ./configure --prefix=$SOS_INSTALL/libevent-2.1.10-stable/install
RUN make clean all install

# Build Hydra
WORKDIR $SOS_INSTALL
RUN wget -c https://www.mpich.org/static/downloads/4.0/hydra-4.0.tar.gz
RUN tar -xzvf hydra-4.0.tar.gz
WORKDIR hydra-4.0
RUN ./autogen.sh
RUN ./configure --prefix=$SOS_INSTALL/hydra-4.0/install
RUN make all install

# Build PMIx
WORKDIR $SOS_INSTALL
RUN git clone -b v4.1.1rc6 --depth 10 https://github.com/pmix/pmix pmix
WORKDIR pmix
RUN ./autogen.pl
RUN ./configure --prefix=$SOS_INSTALL/pmix/install --with-libevent=$SOS_INSTALL/libevent-2.1.10-stable/install --without-libev --disable-debug CFLAGS=-O3 --disable-shared --enable-static
RUN make
RUN make install

# Build PRRTE
WORKDIR $SOS_INSTALL
RUN git clone -b v2.0.0 --depth 10 https://github.com/pmix/prrte prrte
WORKDIR prrte
RUN ./autogen.pl
RUN ./configure --prefix=$SOS_INSTALL/prrte/install --with-pmix=$SOS_INSTALL/pmix/install --without-slurm --with-libevent=$SOS_INSTALL/libevent-2.1.10-stable/install --without-libev --disable-shared --enable-static
RUN make
RUN make install

# Build Libfabric
WORKDIR $SOS_INSTALL
RUN git clone -b v1.7.x --depth 10 https://github.com/ofiwg/libfabric.git libfabric
WORKDIR libfabric
RUN ./autogen.sh
# To build basic Libfabric
RUN ./configure --prefix=$SOS_INSTALL/libfabric/install
# If running with GNI provider:
#RUN ./configure --prefix=$SOS_INSTALL/libfabric/install-gni --enable-gni --disable-verbs --disable-sockets --disable-udp --disable-psm --disable-tcp
RUN make
RUN make install

# Build Portals 4
WORKDIR $SOS_INSTALL
RUN git clone --depth 10 https://github.com/regrant/portals4.git portals4
WORKDIR portals4
RUN ./autogen.sh
RUN ./configure --prefix=$SOS_INSTALL/portals4/install --enable-zero-mrs --enable-reliable-udp --disable-pmi-from-portals
RUN make
RUN make install

# Build UCX
WORKDIR $SOS_INSTALL
RUN git clone -b v1.9.0 --depth 10 https://github.com/openucx/ucx.git
WORKDIR ucx
RUN ./autogen.sh
RUN ./configure --prefix=$SOS_INSTALL/ucx/install --enable-mt --disable-numa --without-java
RUN make
RUN make install

# Build SOS
WORKDIR $SOS_INSTALL
RUN git clone https://github.com/Sandia-OpenSHMEM/SOS.git
WORKDIR SOS
RUN ./autogen.sh
# To Build SOS w/ Basic Libfabric
RUN ./configure --prefix=$SOS_INSTALL/SOS/install --with-ofi=$SOS_INSTALL/libfabric/install --without-ucx --without-portals4 --enable-pmi-simple --disable-fortran
# To Build SOS w/ GNI provider
#RUN ./configure --prefix=$SOS_INSTALL/SOS/install-gni --with-ofi=$SOS_INSTALL/libfabric/install-gni --without-ucx --without-portals4 --enable-pmi-simple --enable-ofi-mr=basic --enable-completion-polling --disable-fortran
# To Build SOS w/ Libfabric & PMIx
#RUN ./configure --prefix=$SOS_INSTALL/SOS/install --with-ofi=$SOS_INSTALL/libfabric/install --without-ucx --without-portals4 --with-pmix=$SOS_INSTALL/pmix/install --disable-shared --enable-static
# To Build SOS w/ Portals 4
#RUN ./configure --prefix=$SOS_INSTALL/SOS/install-portals --with-ofi=$SOS_INSTALL/portals4/install --without-ucx --without-ofi --enable-pmi-simple
# To Build SOS w/ UCX
#RUN ./configure --prefix=$SOS_INSTALL/SOS/install-ucx --with-ucx=$SOS_INSTALL/ucx/install --without-ofi --without-portals4 --enable-pmi-simple
RUN make
RUN make install
RUN make check TESTS= -j

#Start PRRTE Server and Use prun to run tests
#RUN make check TESTS= -j
#ENV PATH=$SOS_INSTALL/prrte/install/bin:$PATH
#RUN prte --daemonize --host localhost:4 --allow-run-as-root
#RUN SHMEM_INFO=1 make VERBOSE=1 TEST_RUNNER="prun -np 2" check
#RUN pterm
76 changes: 76 additions & 0 deletions scripts/docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Sandia-OpenSHMEM Docker Image
Sandia-OpenSHMEM (SOS) utlizes Docker images to provide users a means to create
kholland-intel marked this conversation as resolved.
Show resolved Hide resolved
containers that provide a configurable Ubuntu sandbox for running SOS
applications and benchmarks.

## Creating SOS Containers from the Prebuilt Docker Images
Coming Soon: Information regarding the SOS public Docker Hub repos and prebuilt
images.

## Creating SOS Containers from the Dockerfile
This Dockerfile provides a configurable Ubuntu sandbox for running SOS
applications and benchmarks.

### Building the Image from the Dockerfile
To build the image from the Dockerfile, run the following run the following
kholland-intel marked this conversation as resolved.
Show resolved Hide resolved
command in the directory that contains the Dockerfile:

davidozog marked this conversation as resolved.
Show resolved Hide resolved
```
$ docker build -t <name-of-image> .
```

Where `<name-of-image>` represents the name being used to label the image.
The "docker build" command will use the Dockerfile to build the docker image
from which containers can be created.
To create an interactive container from the image run the following command:

```
$ docker run -it <name-of-image>
```

Where the `-it` option makes the container interactive and allows the user to
actually step into the container and execute commands in the new environment.

### Dockerfile Options
Throughout the Dockerfile there are lines that can be commented/uncommented to
configure a particular testing environment and SOS build.

#### Ubuntu Image
The first option this file provides is the the type of Ubuntu image docker
kholland-intel marked this conversation as resolved.
Show resolved Hide resolved
should pull from.

By default the Dockerfile pulls a lightweight Ubuntu 20.04 image.
The other option (which is commnented out) is for OneAPI image which uses
Ubunutu 18.04. The idea for this image was to allow users to test/run SOS
applications with non gcc compilers.
kholland-intel marked this conversation as resolved.
Show resolved Hide resolved

#### Building Libafabic
davidozog marked this conversation as resolved.
Show resolved Hide resolved
davidozog marked this conversation as resolved.
Show resolved Hide resolved
The next set of options are presented when it comes time to build Libfabric.

By default, a basic libfabric configuration is built and installed to an
kholland-intel marked this conversation as resolved.
Show resolved Hide resolved
"install" directory.
The next option is to build Libfabric with the non-gcc compilers, in this
case, using clang and clang++.
Another option is to enable the GNI provider while building Libfabric.

#### Building SOS
Another set of options are presented when Building SOS.

By default, SOS is built with a basic Libfabric provider.
The next few options are:
* Build SOS with oneapi compilers (i.e. clang and clang++)
* Build SOS with the GNI provider
* Build SOS with the Libfabric provider and PMIx
* Build SOS with Portals 4
* Build SOS with UCX

As a reminder which ever configuration option is chosen must be uncommented,
while the other options must be commented.
kholland-intel marked this conversation as resolved.
Show resolved Hide resolved

#### Enabling the PRRTE Server
The last portion of code that is optional in this Dockerfile is that which
enables the PRRTE Server.
This is mostly used to run SOS tests when the GNI provider has been enabled in
the Libfabric build.
When testing on the GNI provider uncomment out the lines in this portion to
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be better to say the specific line numbers to be uncommented. Same for the previous cases.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of pointing to a specific line number, I suggested user comment out the code directly under a specific subheading (I felt like the headers were less likely to change than the line numbers). Does this work or do you still prefere that we point them to a specific line number? Examples can be found in the README changes at lines 52 and 73.

build a container which runs "make check" on the PRRTE Server.