Update to Spack 0.22.0

[heatherkellyucl]

https://github.com/spack/spack/releases/tag/v0.22.0

The compiler update - adds gcc-runtime and intel-oneapi-runtime packages.
Allows concretised environments to be included in other environments.

• Make a 0.22 branch of this repo
• Move some of the most obvious stuff out into Wiki that should be there rather than in the codebase.
• Update README.md
• Update version and sites_root in spacksites/settings/spack_sites.ini
• Update spacksites/myriad-utilities/git-pull-on-myriad.sh and spacksites/myriad-utilities/init-spacksites-on-myriad.sh
• Update the buildcache location in initial_site_mirrors.yaml
• Check our ucl local package repo to see if specs we got from develop now exist in builtin. If they do, delete the local one.
• Check whether the major changes/deprecations for the new version require any alterations to the spack commands spacksites runs, initial_site_modules.yaml, any other config or format changes or changes in default behaviour.
• Check out the new branch in a new directory, as when starting from scratch above.
• Create the new sites_root in /shared/ucl/apps/$version
• Create a new buildcache in the sites_root, checking whether the versions we build are still available in the new Spack version and updating our site .yaml files if they do not.

Once we think everything is working:

• Make 0.22 the default branch of this repo
• Pull on Myriad
  • Update Myriad buildcache - compiler
  • Update Myriad buildcache - all of initial_set.yaml
• Pull on Kathleen
  • Update Kathleen buildcache - compiler
  • Update Kathleen buildcache - an initial set
• Pull on Young
  • Update Young buildcache - compiler
  • Update Young buildcache - an initial set
• Pull on Michael (remember filesystem will be the same as Young in the end - need to consider old avx2 nodes)
  • Update Michael buildcache - compiler

[giordano]

Check our ucl local package repo to see if specs we got from develop now exist in builtin. If they do, delete the local one.

I'm quite confident they're all in 0.22, as we last touched them 2 months ago, and they were all backports of changes already in spack@develop.

[heatherkellyucl]

• Look at using the binary buildcache for dev tools https://binaries.spack.io/develop/developer-tools-manylinux2014 : it provides common build/dev tools, targeting x86_64_v3

[heatherkellyucl]

Dev notes, questions and initial software list moved out to wiki https://github.com/UCL-ARC/hpc-spack/wiki

[heatherkellyucl]

Right now we have no custom packages in repos/ucl/packages so that directory doesn't exist in the repo and we get this warning (not a problem)

==> Warning: Failed to initialize repository: '$HPC_SPACK_ROOT/repos/ucl'.
  No directory 'packages' found in '$HPC_SPACK_ROOT/repos/ucl'
  To remove the bad repository, run this command:
      spack repo rm $HPC_SPACK_ROOT/repos/ucl

[heatherkellyucl]

Installing first_compiler.yaml, no deprecation issues yet.

[heatherkellyucl]

Built all the things on Myriad (on build01, woo! Except dbcsr), pushing to buildcache:

spack -e myriad buildcache push /shared/ucl/apps/spack/0.22/buildcache

Will make 0.22 the default branch now.

[giordano]

Side note, with spack v0.22 py-alphafold

hpc-spack/spacksites/spack-env-templates/dev1/build/initial_set.yaml

Line 27 in e79971a

# - '[email protected] +cuda cuda_arch=80 ^[email protected] ^[email protected] ^[email protected] +cuda cuda_arch=80'

should finally work

[heatherkellyucl]

spack buildcache list was showing 0 packages, needed to run

spack buildcache update-index /shared/ucl/apps/spack/0.22/buildcache/

[heatherkellyucl]

Have since updated to 0.22.2 and the buildcaches need to be recreated on Kathleen and Young, but we're also wanting to get some things from the manylinux binary buildcache and Owain is going through this in #61 so this issue can be closed.