How to use MPI with solveCRA

Installation of MPI

The MPI prerequisite can be installed with a couple of commands on Ubuntu. The only choice to make is between openmpi and mpich2. Most of our testing is done with openmpi but mpich should also work.

For openmpi do:

$ sudo apt-get install libopenmpi-dev openmpi-bin libhdf5-openmpi-dev

For mpich do:

$ sudo apt-get install mpich libmpich-dev libhdf5-mpich-dev

Please make sure not to install mpich and openmpi together. When both openmpi and mpich are installed strange errors will occur and selalib will not work. If you see both installed please remove both and install one.

Manual install for any linux distribution

Start downloading the last version of OpenMPI from it's official page: http://www.open-mpi.org/software/ompi

Decompress the downloaded file (should be called something like openmpi-x.x.x.tar.xxx, changing x.x.x for the version downloaded):

tar -xvf openmpi-*

Configure the installation file (making use of the superuser privileges of your system) and this task usually takes between 5 and 10 minutes.

It is necessary to add on the prefix the installation directory we want to use for OpenMPI.

The normal thing to do would be to select the next directory “/home//.openmpi”.

$ ./configure --prefix="/home/$USER/.openmpi"

Now is the time for the hard work, install it. For it, we'll make use of the “make” tool. This is a good moment for the coffee, this should take between 10 and 15 minutes, so let it work.

$ make
$ sudo make install

All that is left to do is to include the path to our path environment the path “installation_directory/bin” and to the library environment variable “installation_directory/lib/”. If your system uses bash you'll have to use the command export.

export PATH="$PATH:/home/$USER/.openmpi/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/home/$USER/.openmpi/lib/"

Lauch program compiled with MPI (mpirun(1) man page: https://www.open-mpi.org/doc/v2.0/man1/mpirun.1.php)

The compiled program could be launched as follows without any user provided parameters:

$ mpirun -np 2 name_of_compiled_program

Here -np or -n followed with 2 means to launch 2 processes to run the program, a biger integer could be provided by user if more processes are desired to run with.

If user wish to use the runtime affinity of MPI the an option could be put just before the parameter for number of processes, for example the following will launch the program with 2 processes each of which will be mapped to one core:

$ mpirun --map-by core -np 2 name_of_compiled_program

It is also possible to map one process to one node in the case of launching the program over several nodes of computing centre.

More detail could be found at the following link: https://www.open-mpi.org/doc/v2.0/man1/mpirun.1.php