From 2c71e4fadc5e1e76ca30f6159547e35dfebdfd92 Mon Sep 17 00:00:00 2001 From: Rafael Date: Thu, 22 Feb 2024 18:56:05 +0100 Subject: [PATCH 1/5] updated doc for new Python interface via cmake --- doc/pages/Installation.md | 63 +++++++++++---------------------------- 1 file changed, 18 insertions(+), 45 deletions(-) diff --git a/doc/pages/Installation.md b/doc/pages/Installation.md index eb638449f..1b8272193 100644 --- a/doc/pages/Installation.md +++ b/doc/pages/Installation.md @@ -10,10 +10,9 @@ git clone https://github.com/CRPropa/CRPropa3.git ## Prerequisites + C++ Compiler with C++11 support (gcc, clang and icc are known to work) + Fortran Compiler: to compile SOPHIA -+ numpy: for scientific computations Optionally CRPropa can be compiled with the following dependencies to enable certain functionality. -+ Python and SWIG: to use CRPropa from python (tested for > Python 2.7 and > SWIG 3.0.4) ++ Python, NumPy, and SWIG: to use CRPropa from python (tested for > Python 3.0 and > SWIG 3.0.4) + FFTW3: for turbulent magnetic field grids (FFTW3 with single precision is needed) + Gadget: magnetic fields for large scale structure data + OpenMP: for shared memory parallelization @@ -46,12 +45,12 @@ The following packages are provided with the source code and do not need to be i 2. A set of unit tests can be run with ```make test```. If the tests are successful continue with ```make install``` to install CRPropa at the specified path, or leave it in the build directory. Make sure the - environment variables are set accordingly: E.g. for an installation under - $HOME/.local and using Python 2.7 set + environment variables are set accordingly: e.g. for an installation under + $HOME/.local and using Python 3 set ```sh export PATH=$HOME/.local/bin:$PATH export LD_LIBRARY_PATH=$HOME/.local/lib:$LD_LIBRARY_PATH - export PYTHONPATH=$HOME/.local/lib/python2.7/site-packages:$PYTHONPATH + export PYTHONPATH=$HOME/.local/lib/python3.9/site-packages:$PYTHONPATH export PKG_CONFIG_PATH=$HOME/.local/lib/pkgconfig:$PKG_CONFIG_PATH ``` @@ -110,7 +109,7 @@ worthwhile effort afterwards. To install python dependencies and libraries use `pip`. Example: `pip install numpy`. -4. Compile and install CRPropa (please note specific [insturctions for different operating systems](#notes-for-specific-operating-systems)). +4. Compile and install CRPropa (please note specific [instructions for different operating systems](#notes-for-specific-operating-systems)). ```sh cd $CRPROPA_DIR git clone https://github.com/CRPropa/CRPropa3.git @@ -144,7 +143,7 @@ cmake -DENABLE_PYTHON=ON .. ``` + Set the install path ```-DCMAKE_INSTALL_PREFIX=/my/install/path``` -+ Enable Galactic magnetic lens ```-DENABLE_GALACTICMAGETICLENS=ON``` ++ Enable Galactic magnetic lens ```-DENABLE_GALACTICMAGNETICLENS=ON``` + Enable FFTW3 (turbulent magnetic fields) ```-DENABLE_FFTW3F=ON``` + Enable OpenMP (multi-core parallel computing) ```-DENABLE_OPENMP=ON``` + Enable Python (Python interface with SWIG) ```-DENABLE_PYTHON=ON``` @@ -178,11 +177,13 @@ cmake -DENABLE_PYTHON=ON .. + Quite often there are multiple Python versions installed in a system. This is likely the cause of many (if not most) of the installation problems related to Python. To prevent conflicts among them, one can explicitly refer to the Python version to be used. Example: ``` - -DCMAKE_PYTHON_EXECUTABLE=/usr/bin/python - -DCMAKE_PYTHON_INCLUDE_DIR= - -DCMAKE_PYTHON_LIBRARY=/libpython.so + -DPython_EXECUTABLE=/usr/bin/python + -DPython_INCLUDE_DIRS= + -DPython_LIBRARY=/libpython.so ``` Note that in systems running OSX, the extension .so should be replaced by .dylib. + For further details, see [FindPython.cmake](https://cmake.org/cmake/help/latest/module/FindPython.html#module:FindPython). + ## Notes for Specific Operating Systems @@ -201,41 +202,13 @@ For Fedora/CentOS/RHEL the required packages to build CRPropa: ``` In case of CentOS/RHEL 7, the SWIG version is too old and has to be built from source. - ### Mac OS X -Tested on version 12.5.1 with M1 pro where command line developer tools are installed. -Install Python3, and llvm from Homebrew, and specify the following paths to the Python and llvm directories in the Homebrew folder after step 3 of the above installation, e.g. (please use your exact versions): - ```sh - export LLVM_DIR="/opt/homebrew/Cellar/llvm/15.0.7_1" - PYTHON_VERSION=3.10 - LLVM_VERSION=15.0.7 - PYTHON_DIR=/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10 - ``` -and replace the command in step 4 of the installation routine - ```sh - CMAKE_PREFIX_PATH=$CRPROPA_DIR cmake -DCMAKE_INSTALL_PREFIX=$CRPROPA_DIR .. - ``` -with - ```sh - cmake .. \ - -DCMAKE_INSTALL_PREFIX=$CRPROPA_DIR \ - -DPYTHON_EXECUTABLE=$PYTHON_DIR/bin/python$PYTHON_VERSION \ - -DPYTHON_LIBRARY=$PYTHON_DIR/lib/libpython$PYTHON_VERSION.dylib \ - -DPYTHON_INCLUDE_PATH=$PYTHON_DIR/include/python$PYTHON_VERSION \ - -DCMAKE_C_COMPILER=$LLVM_DIR/bin/clang \ - -DCMAKE_CXX_COMPILER=$LLVM_DIR/bin/clang++ \ - -DOpenMP_CXX_FLAGS="-fopenmp -I$LLVM_DIR/lib/clang/$LLVM_VERSION/include" \ - -DOpenMP_C_FLAGS="-fopenmp =libomp -I$LLVM_DIR/lib/clang/$LLVM_VERSION/include" \ - -DOpenMP_libomp_LIBRARY=$LLVM_DIR/lib/libomp.dylib \ - -DCMAKE_SHARED_LINKER_FLAGS="-L$LLVM_DIR/lib -lomp -Wl,-rpath,$LLVM_DIR/lib" \ - -DOpenMP_C_LIB_NAMES=libomp \ - -DOpenMP_CXX_LIB_NAMES=libomp \ - -DNO_TCMALLOC=TRUE +For a clean OS X (Sonoma 14+) installation, if you use Homebrew, the main dependencies can be installed as follows: + ```sh + brew install hdf5 fftw cfitsio muparser libomp numpy swig ``` -Check that all paths are set correctly with the following command in the build folder - ```sh - ccmake .. +Similarly, if you use MacPorts instead of Homebrew, download the corresponding packages: + ```sh + sudo port install hdf5 fftw cfitsio muparser libomp numpy swig ``` -and configure and generate again after changes. - - +Note that if you are using a Mac with Arm64 architecture (M1, M2, or M3 processors), `SIMD_EXTENSIONS` might not run straight away. From 30644625918b7daefcf1ae5c98f9d534e250cacd Mon Sep 17 00:00:00 2001 From: Rafael Date: Fri, 23 Feb 2024 13:05:11 +0100 Subject: [PATCH 2/5] add information on Python_INSTALL_PACKAGE_DIR --- doc/pages/Installation.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/doc/pages/Installation.md b/doc/pages/Installation.md index 1b8272193..4b2c9ef95 100644 --- a/doc/pages/Installation.md +++ b/doc/pages/Installation.md @@ -181,8 +181,13 @@ cmake -DENABLE_PYTHON=ON .. -DPython_INCLUDE_DIRS= -DPython_LIBRARY=/libpython.so ``` - Note that in systems running OSX, the extension .so should be replaced by .dylib. - For further details, see [FindPython.cmake](https://cmake.org/cmake/help/latest/module/FindPython.html#module:FindPython). +Note that in systems running OSX, the extension .so should be replaced by .dylib. +In addition, The path where the CRPropa python module is installed can be specified with the flag: +``` +-DPython_INSTALL_PACKAGE_DIR= +``` +For further details, see [FindPython.cmake](https://cmake.org/cmake/help/latest/module/FindPython.html#module:FindPython). + ## Notes for Specific Operating Systems From 9698b24d4ec149f9227a829e4ca216ed1eba4d74 Mon Sep 17 00:00:00 2001 From: Rafael Date: Fri, 23 Feb 2024 13:06:47 +0100 Subject: [PATCH 3/5] update required swig version --- doc/pages/Installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/pages/Installation.md b/doc/pages/Installation.md index 4b2c9ef95..177a62195 100644 --- a/doc/pages/Installation.md +++ b/doc/pages/Installation.md @@ -12,7 +12,7 @@ git clone https://github.com/CRPropa/CRPropa3.git + Fortran Compiler: to compile SOPHIA Optionally CRPropa can be compiled with the following dependencies to enable certain functionality. -+ Python, NumPy, and SWIG: to use CRPropa from python (tested for > Python 3.0 and > SWIG 3.0.4) ++ Python, NumPy, and SWIG: to use CRPropa from python (tested for > Python 3.0 and > SWIG 4.0.2) + FFTW3: for turbulent magnetic field grids (FFTW3 with single precision is needed) + Gadget: magnetic fields for large scale structure data + OpenMP: for shared memory parallelization From be12d2fa25a77fd71c9aaed7f04c43cbe3f550d2 Mon Sep 17 00:00:00 2001 From: Rafael Date: Fri, 23 Feb 2024 13:10:57 +0100 Subject: [PATCH 4/5] keep prvious instructions for OSX --- doc/pages/Installation.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/doc/pages/Installation.md b/doc/pages/Installation.md index 177a62195..c94551ae0 100644 --- a/doc/pages/Installation.md +++ b/doc/pages/Installation.md @@ -217,3 +217,42 @@ Similarly, if you use MacPorts instead of Homebrew, download the corresponding p sudo port install hdf5 fftw cfitsio muparser libomp numpy swig ``` Note that if you are using a Mac with Arm64 architecture (M1, M2, or M3 processors), `SIMD_EXTENSIONS` might not run straight away. + + +Some combinations of versions of the Apple's clang compiler and python might lead to installation errors. +In these cases, the user might want to consider the workaround below (tested on version 12.5.1 with M1 pro where command line developer tools are installed). + +Install Python3, and llvm from Homebrew, and specify the following paths to the Python and llvm directories in the Homebrew folder after step 3 of the above installation, e.g. (please use your exact versions): + ```sh + export LLVM_DIR="/opt/homebrew/Cellar/llvm/15.0.7_1" + PYTHON_VERSION=3.10 + LLVM_VERSION=15.0.7 + PYTHON_DIR=/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10 + ``` +and replace the command in step 4 of the installation routine + ```sh + CMAKE_PREFIX_PATH=$CRPROPA_DIR cmake -DCMAKE_INSTALL_PREFIX=$CRPROPA_DIR .. + ``` +with + ```sh + cmake .. \ + -DCMAKE_INSTALL_PREFIX=$CRPROPA_DIR \ + -DPython_EXECUTABLE=$PYTHON_DIR/bin/python$PYTHON_VERSION \ + -DPython_LIBRARY=$PYTHON_DIR/lib/libpython$PYTHON_VERSION.dylib \ + -DPython_INCLUDE_PATH=$PYTHON_DIR/include/python$PYTHON_VERSION \ + -DCMAKE_C_COMPILER=$LLVM_DIR/bin/clang \ + -DCMAKE_CXX_COMPILER=$LLVM_DIR/bin/clang++ \ + -DOpenMP_CXX_FLAGS="-fopenmp -I$LLVM_DIR/lib/clang/$LLVM_VERSION/include" \ + -DOpenMP_C_FLAGS="-fopenmp =libomp -I$LLVM_DIR/lib/clang/$LLVM_VERSION/include" \ + -DOpenMP_libomp_LIBRARY=$LLVM_DIR/lib/libomp.dylib \ + -DCMAKE_SHARED_LINKER_FLAGS="-L$LLVM_DIR/lib -lomp -Wl,-rpath,$LLVM_DIR/lib" \ + -DOpenMP_C_LIB_NAMES=libomp \ + -DOpenMP_CXX_LIB_NAMES=libomp \ + -DNO_TCMALLOC=TRUE + ``` +Check that all paths are set correctly with the following command in the build folder + ```sh + ccmake .. + ``` +and configure and generate again after changes. + From b005f9aaee3a639b3f899d5111354b86aeba1975 Mon Sep 17 00:00:00 2001 From: Rafael Date: Wed, 28 Feb 2024 12:14:11 +0100 Subject: [PATCH 5/5] updated tested python versions --- doc/pages/Installation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/pages/Installation.md b/doc/pages/Installation.md index c94551ae0..c9f69ead5 100644 --- a/doc/pages/Installation.md +++ b/doc/pages/Installation.md @@ -12,7 +12,7 @@ git clone https://github.com/CRPropa/CRPropa3.git + Fortran Compiler: to compile SOPHIA Optionally CRPropa can be compiled with the following dependencies to enable certain functionality. -+ Python, NumPy, and SWIG: to use CRPropa from python (tested for > Python 3.0 and > SWIG 4.0.2) ++ Python, NumPy, and SWIG: to use CRPropa from python (tested for >= Python 3.7 and > SWIG 4.0.2) + FFTW3: for turbulent magnetic field grids (FFTW3 with single precision is needed) + Gadget: magnetic fields for large scale structure data + OpenMP: for shared memory parallelization