Traverse @ Princeton

Instructions to install simsopt on Traverse

Traverse has a low-endian PowerPC64 architecture. Since Github runners don't have the required hardware, we can not release the PPC64 wheels and containers along with those of X86_64 and M1. We hope this will serve you a guide for installing simsopt on other PowerPC64 machines.

Environment Setup

1. Set up the environment based on conda. To do that load modules for anaconda.

   module load anaconda3/2022.5
   

2. Create a conda virtual environment with python 3.9. Here we use simsopt as the name for the virtual environment.

   conda create -n simsopt python=3.9
   

3. Before installing packages, load the compilers.

   module load openmpi/cuda-11.0/gcc/4.0.4/64
   

4. To minimize package mismatches with conda, we will try to install many of the dependencies from conda instead of from pip.

   conda install numpy scipy Deprecated nptyping  monty ruamel.yaml sympy f90nml randomgen pyevtk matplotlib
   

   If you can not install any of these packages from conda, try pip later.
5. We need other packages such as JAX and mpi4py, which we can install only from pip or manually. There is a conda package for mpi4py, but don't use it. Installing mpi4py from conda installs MPI compilers that clash with system MPI.

   pip install --no-cache-dir mpi4py
   

6. Jax can only be installed manually on PowerPC. The version is also not the latest, but an older one released in 2021. We also need to load CUDA modules for Jax. Credits to Rory from DESC development team.

   module load cudatoolkit/11.1 cudnn/cuda-11.1/8.0.4
   

   Download jax source code

   git clone https://github.com/google/jax.git
   cd jax
   
   # last commit of JAX that we got to work with Traverse
   git checkout 6c08702489b33f6c51d5cf0ccadc45e997ab406e
   
   python build/build.py --enable_cuda --cuda_path /usr/local/cuda-11.1 --cuda_version=11.1 --cudnn_version=8.0.4 --cudnn_path /usr/local/cudnn/cuda-11.1/8.0.4 --noenable_mkl_dnn --bazel_path /usr/bin/bazel --target_cpu=ppc
   pip install dist/*.whl
   pip install .
   

7. Now is the time to install simsopt. Download the source code.

   git clone https://github.com/hiddensymmetries/simsopt.git
   cd simsopt
   

   Apply the patch for PowerPC

   git apply patches/simsopt_ppc.patch

   Install with pip

   pip install .