-
Notifications
You must be signed in to change notification settings - Fork 78
cgdm hearhear
- Overview
- Installing spack
-
Installing software with spack
- General notes
- Installing lmod
- Setting up module-related defaults for spack
- Installing gcc and gfortran
- Installing openmpi and mpich
- (Re)creating the module files
- Making a module file for clang-gfortran
- Installing NetCDF
- Installing some python things
- Installing the OpenMP libraries
- Installing other convenience software
- Installing conda
- Known issues
cgdm-hearhear is an M2-based Mac laptop that the ESMF team set up November, 2024. It is set up with spack and modules to facilitate testing multiple compilers and MPI libraries.
This wiki page documents how this machine was set up. Note that the spack installation is currently under Bill Sacks's home directory (/Users/sacks/spack
), but this could be redone in a shared space if desired.
The steps documented here are skipping some instances of undoing / redoing things as I was trying to get things working. The steps I'm leaving out are ones that I expect did not impact the end result in any significant way.
The starting point for this was essentially a fresh machine: the Apple developer tools were installed, but there was nothing installed with either Homebrew or MacPorts.
I installed spack in /Users/sacks/spack
via:
git clone -c feature.manyFiles=true --depth=2 https://github.com/spack/spack.git
cd spack
git fetch --tags
git checkout develop-2024-11-10
I then added the following to my .bash_profile
:
. ${HOME}/spack/share/spack/setup-env.sh
I'm generally building the software stack with clang, only building things with gcc where needed, since I ran into trouble trying to build some lower-level libraries with gcc.
The latest (8.7.37) seems to have a bug: the init files are empty, leading to a situation where the "module" command isn't available; so using the older 8.7.24.
spack install [email protected]
Put the following in .bash_profile
after the spack setup line:
. $(spack location -i lmod)/lmod/lmod/init/bash
And this, to use the version of modules for hierarchical modules:
module use ${SPACK_ROOT}/share/spack/lmod/darwin-sonoma-aarch64/Core/
(This directory may not exist yet, but will soon.)
And also creating a space for my own personal modules:
module use ${HOME}/modules
So my full .bash_profile
now looks like this:
# Spack stuff
. ${HOME}/spack/share/spack/setup-env.sh
. $(spack location -i lmod)/lmod/lmod/init/bash
module use ${SPACK_ROOT}/share/spack/lmod/darwin-sonoma-aarch64/Core/
module use ${HOME}/modules
I created the following file in ~/.spack/modules.yaml
:
modules:
default:
enable: [lmod]
lmod:
hash_length: 0
exclude_implicits: true
exclude:
- lmod
include:
- cmake
- python
- py-pip
- py-pyyaml
core_compilers:
- apple-clang@=16.0.0
core_specs:
- cmake
- python
- py-pip
- py-pyyaml
hierarchy:
- mpi
projections:
netcdf-c^mpi: '{name}/{version}'
netcdf-c: '{name}-nompi/{version}'
netcdf-fortran^mpi: '{name}/{version}'
netcdf-fortran: '{name}-nompi/{version}'
Notes on this:
-
enable: [lmod]
enables the creation of module files for packages that spack installs. (The spack documentation suggests runningspack config add modules:default:enable:[lmod]
, which creates this setting.) -
hash_length: 0
removes the hash from the module names (see https://spack-tutorial.readthedocs.io/en/latest/tutorial_modules.html#change-module-file-naming); with the other changes here for hierarchical modules, this hash is unnecessary. -
exclude_implicits: true
avoids creating modules for packages that are installed as dependencies, instead only creating modules for packages that are explicitly installed withspack install
. This solves three issues:- It avoids listing a bunch of lower-level, uninteresting modules when running
module avail
. (This can also be achieved withhide_implicits: true
, but usinghide_implicits
instead ofexclude_implicits
leaves the following issues unresolved.) - It avoids listing the lower-level, uninteresting modules when running
module list
after doing amodule load
of a module that depends on some of these lower-level modules. - Perhaps most importantly, it avoids issues I was running into with auto-loading lower-level modules that were created with a different compiler -- specifically, loading the module of a library that was created with gcc/gfortran but which depended on libraries created with clang/gfortran. (I think there may have been inconsistencies between the module paths to these lower-level libraries in the module files for the dependent libraries, and the paths where these lower-level module files actually resided. It's possible that there would be a way to resolve this issue, but simply avoiding creating modules at all for these lower-level libraries avoided the problem.)
- It avoids listing a bunch of lower-level, uninteresting modules when running
-
exclude: lmod
avoids creating a module file forlmod
, since that seems unnecessary and kind of circular. - The
include
list will come into play later, after we install more things: This lists some implicitly-installed packages for which we do want modules. -
core_compilers: apple-clang@=16.0.0
specifies that anything built with this core (i.e., system-level) compiler should appear in the 'Core' part of the module hierarchy and thus always available, rather than being dependent on loading any particular compiler module. - The
core_specs
list, like theinclude
list, will come into play later, after we install more things: This lists some packages that we want to appear in the compiler-independentCore
modules despite having been built with a non-core compiler. -
hierarchy: mpi
says to include the mpi library in the module hierarchy. So modules for libraries that depend on a given mpi library will only appear once you load that mpi library. - The
projections
section with itsnetcdf
specifications will become relevant once we install NetCDF in a future step, below. The important aspect of this is to specify a naming convention that distinguishes between mpi and no-mpi versions of netcdf. Without this, after loading a compiler and mpi library, there would be two versions of the netcdf modules available with the same name: one built with the mpi library and one built without mpi. The settings here use the default naming convention ({name}/{version}
) for the netcdf libraries built with dependence on an mpi library, but add-nompi
to the name for netcdf libraries built without this dependence on an mpi library.
The latest gcc has problems building on MacOS (https://github.com/spack/spack/issues/45628), so using version 14.1.0:
spack install [email protected]
spack load gcc
spack compiler find
I then needed to fix the compiler spec for spack's clang-gfortran combination: In ~/.spack/darwin/compilers.yaml
, there are two blocks with spec: apple-clang@=16.0.0
: one has f77 and fc set to null; the second has them set to the gcc-14.1.0 gfortran. I'm changing the spec name for the second to be apple-clang@=16.0.0-gfortran14.1.0
. (See also https://spack-tutorial.readthedocs.io/en/latest/tutorial_configuration.html#compiler-configuration).
First installing these mpi libraries with clang-gfortran:
spack install openmpi %apple-clang@=16.0.0-gfortran14.1.0
spack install mpich %apple-clang@=16.0.0-gfortran14.1.0
And then with gcc; note that these installations reused the dependencies installed using clang and just built the mpi libraries themselves using gcc. (This is important because some of the dependencies have build issues using gcc.):
spack install openmpi %[email protected]
spack install mpich %[email protected]
At this point and a few others, I had spack recreate its module files:
spack module lmod refresh --delete-tree -y
Since the clang-gfortran compiler combination wasn't something that spack installed, spack doesn't create a module file for it. We need a module file for two purposes:
- Loading the necessary gcc/gfortran stuff when using clang-gfortran. (An alternative would be to load the gcc module, since we need to do most of the same stuff for both, and then set/unset environment variable(s) like CC. But that wouldn't accomplish the second purpose.)
- Getting the clang-gfortran modules (like openmpi and mpich) to appear when we're using clang-gfortran, given the hierarchical module setup we're using.
I created a module for this, which I'm putting in my home directory, not under spack, so that when I ask spack to rebuild modules, it doesn't mess with this one. So creating file ${HOME}/modules/apple-clang/16.0.0-gfortran14.1.0.lua
. This started as a copy of gcc/14.1.0.lua
, but then I made some changes:
--- 14.1.0.lua 2024-11-15 14:05:52
+++ 16.0.0-gfortran14.1.0.lua 2024-11-15 14:05:48
@@ -1,27 +1,26 @@
-- -*- lua -*-
--- Module file created by spack (https://github.com/spack/spack) on 2024-11-15 13:48:15.463589
+-- Module file created by modifying the gcc file created by spack (https://github.com/spack/spack) on 2024-11-14 11:44:29.184355
--
-- [email protected]%[email protected]~binutils~bootstrap~graphite~mold~nvptx~piclibs~strip build_system=autotools build_type=RelWithDebInfo languages='c,c++,fortran' patches=1529cff arch=darwin-sonoma-m2/yxln5vj
--
-whatis([[Name : gcc]])
-whatis([[Version : 14.1.0]])
+whatis([[Name : apple-clang]])
+whatis([[Version : 16.0.0-gfortran14.1.0]])
whatis([[Target : m2]])
-whatis([[Short description : The GNU Compiler Collection includes front ends for C, C++, Objective-C, Fortran, Ada, and Go, as well as libraries for these languages.]])
+whatis([[Short description : Use built-in clang for C/C++, and gfortran for Fortran.]])
whatis([[Configure options : --with-pkgversion=Spack GCC --with-bugurl=https://github.com/spack/spack/issues --disable-multilib --enable-languages=c,c++,fortran --disable-nls --disable-canonical-system-headers --with-system-zlib --with-zstd-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/include --with-zstd-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/lib --with-ld=/Library/Developer/CommandLineTools/usr/bin/ld-classic --disable-bootstrap --with-mpfr-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/include --with-mpfr-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/lib --with-gmp-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/include --with-gmp-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/lib --with-mpc-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/include --with-mpc-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/lib --without-isl --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk --with-libiconv-prefix=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/libiconv-1.17-6735mlktagraawr36ttz4rzm6i5p6lis --with-stage1-ldflags=-Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib64 -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/libiconv-1.17-6735mlktagraawr36ttz4rzm6i5p6lis/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zlib-ng-2.2.1-n5joj3sdmckf3f545vscegafl34xo6rz/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/lib --with-boot-ldflags=-Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib64 -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/libiconv-1.17-6735mlktagraawr36ttz4rzm6i5p6lis/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zlib-ng-2.2.1-n5joj3sdmckf3f545vscegafl34xo6rz/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/lib -static-libstdc++ -static-libgcc --with-build-config=spack]])
-help([[Name : gcc]])
-help([[Version: 14.1.0]])
+help([[Name : apple-clang]])
+help([[Version: 16.0.0-gfortran14.1.0]])
help([[Target : m2]])
help()
-help([[The GNU Compiler Collection includes front ends for C, C++, Objective-C,
-Fortran, Ada, and Go, as well as libraries for these languages.]])
+help([[Use built-in clang for C/C++, and gfortran for Fortran.]])
-- Services provided by the package
family("compiler")
-- Loading this module unlocks the path below unconditionally
-prepend_path("MODULEPATH", "/Users/sacks/spack/share/spack/lmod/darwin-sonoma-aarch64/gcc/14.1.0")
+prepend_path("MODULEPATH", "/Users/sacks/spack/share/spack/lmod/darwin-sonoma-aarch64/apple-clang/16.0.0-gfortran14.1.0")
@@ -29,7 +28,7 @@
prepend_path("PATH", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/bin", ":")
prepend_path("MANPATH", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/share/man", ":")
prepend_path("CMAKE_PREFIX_PATH", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/.", ":")
-setenv("CC", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/bin/gcc")
+setenv("CC", "/usr/bin/clang")
setenv("FC", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/bin/gfortran")
setenv("F77", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/bin/gfortran")
append_path("MANPATH", "", ":")
Resulting in this file:
-- -*- lua -*-
-- Module file created by modifying the gcc file created by spack (https://github.com/spack/spack) on 2024-11-14 11:44:29.184355
--
-- [email protected]%[email protected]~binutils~bootstrap~graphite~mold~nvptx~piclibs~strip build_system=autotools build_type=RelWithDebInfo languages='c,c++,fortran' patches=1529cff arch=darwin-sonoma-m2/yxln5vj
--
whatis([[Name : apple-clang]])
whatis([[Version : 16.0.0-gfortran14.1.0]])
whatis([[Target : m2]])
whatis([[Short description : Use built-in clang for C/C++, and gfortran for Fortran.]])
whatis([[Configure options : --with-pkgversion=Spack GCC --with-bugurl=https://github.com/spack/spack/issues --disable-multilib --enable-languages=c,c++,fortran --disable-nls --disable-canonical-system-headers --with-system-zlib --with-zstd-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/include --with-zstd-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/lib --with-ld=/Library/Developer/CommandLineTools/usr/bin/ld-classic --disable-bootstrap --with-mpfr-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/include --with-mpfr-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/lib --with-gmp-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/include --with-gmp-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/lib --with-mpc-include=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/include --with-mpc-lib=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/lib --without-isl --with-native-system-header-dir=/usr/include --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk --with-libiconv-prefix=/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/libiconv-1.17-6735mlktagraawr36ttz4rzm6i5p6lis --with-stage1-ldflags=-Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib64 -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/libiconv-1.17-6735mlktagraawr36ttz4rzm6i5p6lis/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zlib-ng-2.2.1-n5joj3sdmckf3f545vscegafl34xo6rz/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/lib --with-boot-ldflags=-Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib64 -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gmp-6.3.0-xt3cjmh7dehpxzqwjklftlml6vdqica2/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/libiconv-1.17-6735mlktagraawr36ttz4rzm6i5p6lis/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpc-1.3.1-r4htrqxnrh4ia7hj5na3uxxihujzcgqf/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/mpfr-4.2.1-k6l2mohdprx77kyqgknyfgyyv7tfi5nh/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zlib-ng-2.2.1-n5joj3sdmckf3f545vscegafl34xo6rz/lib -Wl,-rpath,/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/zstd-1.5.6-qgzwdavw3b3q35p2jt7qu4yb5kmkbskr/lib -static-libstdc++ -static-libgcc --with-build-config=spack]])
help([[Name : apple-clang]])
help([[Version: 16.0.0-gfortran14.1.0]])
help([[Target : m2]])
help()
help([[Use built-in clang for C/C++, and gfortran for Fortran.]])
-- Services provided by the package
family("compiler")
-- Loading this module unlocks the path below unconditionally
prepend_path("MODULEPATH", "/Users/sacks/spack/share/spack/lmod/darwin-sonoma-aarch64/apple-clang/16.0.0-gfortran14.1.0")
prepend_path("DYLD_FALLBACK_LIBRARY_PATH", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/lib", ":")
prepend_path("PATH", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/bin", ":")
prepend_path("MANPATH", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/share/man", ":")
prepend_path("CMAKE_PREFIX_PATH", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/.", ":")
setenv("CC", "/usr/bin/clang")
setenv("FC", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/bin/gfortran")
setenv("F77", "/Users/sacks/spack/opt/spack/darwin-sonoma-m2/apple-clang-16.0.0/gcc-14.1.0-yxln5vjswuutfxtzpq4tf5yk7iuhin4a/bin/gfortran")
append_path("MANPATH", "", ":")
Note that, when installing new versions of gcc/gfortran, or doing something that causes significant changes in the spack-generated module files, I'll need to recreate this manually-created module file.
First, with clang-gfortran with openmpi:
spack install netcdf-c %[email protected] ^openmpi
spack install netcdf-fortran %[email protected] ^openmpi
and with mpich (note that this also installs a new hdf5, since hdf5 depends on the mpi library):
spack install netcdf-c %[email protected] ^mpich
spack install netcdf-fortran %[email protected] ^mpich
and without mpi (note that we need to explicitly specify ^hdf5~mpi
to avoid reusing an already-built hdf5 library that depends on mpi; this forces creating a new version of hdf5 without mpi):
spack install netcdf-c %[email protected] ~mpi ^hdf5~mpi
spack install netcdf-fortran %[email protected] ^netcdf-c~mpi
And then similar for gcc (note that I needed to explicitly ask for versions of dependencies that were built using gcc -- otherwise, spack tried to use dependencies like openmpi that were built using clang-gfortran; this caused problems in the paths where the netcdf modules were created, and might have caused other problems as well):
spack install netcdf-c %[email protected] ^openmpi%[email protected]
spack install netcdf-fortran %[email protected] ^openmpi%[email protected]
spack install netcdf-c %[email protected] ^mpich%[email protected]
spack install netcdf-fortran %[email protected] ^mpich%[email protected]
spack install netcdf-c %[email protected] ~mpi ^hdf5%[email protected]~mpi
spack install netcdf-fortran %[email protected] ^netcdf-c%[email protected]~mpi
Python itself was already installed earlier (as a dependency of something else), but the ESMF testing needs some other things:
spack install py-pip
spack install py-pyyaml
This led to duplications of py-pip; I got rid of one with spack uninstal /bqgtfvr
(to uninstall the one I had installed manually, just leaving the one that got installed in the course of installing pyyaml) (so maybe the first spack install py-pip
was unnecessary).
Note that I now have two python modules, but the py-pip and py-pyyaml modules are built on python/3.11.9, so that one gets loaded when I load py-pip or py-pyyaml.
Apple's clang doesn't have support for OpenMP. To get this support we need to install the OpenMP libraries, so that the necessary libraries and header files are available somewhere:
spack install llvm-openmp %apple-clang@=16.0.0-gfortran14.1.0
Installed emacs with:
spack install emacs
Then added emacs
to the core_specs
section of .spack/modules.yaml
.
I had trouble installing conda through spack (I tried both miniconda3 and anaconda3):
Error: failed to concretize `miniconda3` for the following reasons:
1. Cannot select a single "version" for package "miniconda3"
(specifying a specific version with @
doesn't solve this).
So instead installing miniconda via https://docs.anaconda.com/miniconda/install/#quick-command-line-install
Then ran:
source ~/miniconda3/bin/activate
conda init --all
I'm also loading the necessary python modules in my .bash_profile
. (We can't load them in the ESMF test scripts, because we need pyyaml available at the time when we start up these scripts.)
So my .bash_profile
now looks like this:
# Spack stuff
. ${HOME}/spack/share/spack/setup-env.sh
. $(spack location -i lmod)/lmod/lmod/init/bash
module use ${SPACK_ROOT}/share/spack/lmod/darwin-sonoma-aarch64/Core/
module use ${HOME}/modules
# Load some modules
module load py-pip/23.1.2 py-pyyaml/6.0.2 python/3.11.9
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/Users/sacks/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/Users/sacks/miniconda3/etc/profile.d/conda.sh" ]; then
. "/Users/sacks/miniconda3/etc/profile.d/conda.sh"
else
export PATH="/Users/sacks/miniconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<
Also running:
conda config --set auto_activate_base false
When building an application with a mix of gfortran and clang, where clang is used as the final linker, under some circumstances clang produces an error, duplicate LC_RPATH
. Specifically, I have encountered this when an intermediate library is created with gfortran and then clang is used to link this library with other libraries and/or object files. e.g., see https://github.com/spack/spack/issues/46506#issuecomment-2492650588.
In ESMF testing, this only shows up when using gfortranclang with openmpi (because, with this combination, the trace-related library is linked with the Fortran compiler rather than the C++ compiler, but then the final application is linked using the C++ compiler, as is the case in general with gfortranclang). For now I am avoiding this issue by setting ESMF_TRACE_LIB_BUILD=OFF
for this combination.
- Repository
- Documentation
- Testing
- Release Procedure
- Updating Copyright Year
- ESMF-managed machines
- ESMF web site
- Application Testing