Skip to content

Sync meeting on EESSI software layer (2023 05 30)

Jump to bottom
Kenneth Hoste edited this page May 31, 2023 · 1 revision

EESSI/2023.04 software layer

(copied from https://hackmd.io/58wNKPaCSSS97D7U-5KB4g)

This document collects information about the EESSI/2023.04 software-layer (what toolchains and software packages it should include, how to build the software, what issues are encountered and how they have been resolved). It also serves to coordinate the building of the software layer version and keeps track of meetings/discussions around that.

Meeting 2023-05-30, 09:00 CEST (UTC+2)

Attending:

  • status EESSI/2023.04
    • compilers GCC/10.3.0, GCC/11.3.0 & GCC/12.2.0 can be built
      • requires just merged EESSI/software-layer PR#251 hook to create RPATH wrapper for system type prefix like 'x86_64-pc-linux-gnu'
      • should use EasyBuild 4.7.2
      • bot PR #172 should be merged
        • should we disable use of bot: build label?
        • is equivalent with bot: build comment, which gives more finegrained control
        • should be done in a separate PR
  • status NESSI/2023.04 
    • eb --missing foss-2021a.eb
[snip]
7 out of 37 required modules missing:

* gompi/2021a (gompi-2021a.eb)
* FFTW/3.3.9-gompi-2021a (FFTW-3.3.9-gompi-2021a.eb)
* BLIS/0.8.1-GCC-10.3.0 (BLIS-0.8.1-GCC-10.3.0.eb)
* FlexiBLAS/3.0.4-GCC-10.3.0 (FlexiBLAS-3.0.4-GCC-10.3.0.eb)
* ScaLAPACK/2.1.0-gompi-2021a-fb (ScaLAPACK-2.1.0-gompi-2021a-fb.eb)
* foss/2021a (foss-2021a.eb)
      except for foss/2021a we should be done with the toolchain after https://github.com/NorESSI/software-layer/pull/111 has been ingested
    • issues encountered + fixes
  • current work for NESSI/2023.04
    • trying to build additional packages
  • notes:
  • Alan: could implement policy via EasyBuild parse hook that gets run in GitHub Actions
  • what about license checks
    • "license_spdx = 'GPLv2'" support in EasyBuild
    • using tools like https://github.com/licensee/licensee
    • whitelist we maintain in EESSI/software-layer (in a hook)
    • GitHub Actions can set status on commit based on whether all missing software is whitelisted (or has license_spdx set)
  • we should only use merged PRs
  • separate easystack files
    • eb-4.7.2_2020b.yml
    • for generic builds
      • eb-4.7.2_2020b_arch-generic.yml
    • x86_64
      • eb-4.7.2_2020b_arch-x86_64.yml
    • need to make sure that order in which installations are done is consistent
    • currently we need to handle special cases in the install script, like easyblock required for OpenBLAS
    • could try to work to a single collapsed easystack file:
    easyconfigs:
    - GCC:
        - version: 10.3.0
    - OpenBLAS-0.3.20-GCCcore-10.3.0:
            - arch=*/generic:
                - include-easyblocks-from-pr: 123
            - arch=aarch64/*:
                - SKIP
                - hooks: fake_module_hook.py
            - arch=all:
    - foss-2020b:
    • the call of check_missing_installations.sh in EESSI-pilot-install-software.sh should be run for specific easystack files
      • currently it uses eessi-EESSI_PILOT_VERSION.yml
  • we could do a test build in GitHub Actions (Alan?)
    • can also check for missing installations here
  • should also look into bwrap vs fuse-overlayfs
  • next steps
    • [Kenneth] configure bots in AWS, Azure
      • bot: build should only build stuff we need for EESSI/2023.04
    • [Kenneth] EasyBuild v4.7.2
    • separate PRs for diff. GCC versions:
      • [Kenneth?] 9.3.0 (broken! - for foss/2020b - not a priority)
      • [Thomas] 10.3.0 (foss/2021a)
        • plus write up some instructions
      • [Lara] 11.2.0 (foss/2021b)
      • [Alan] 11.3.0 (for foss/2022a)
      • [Jure] 12.2.0 (foss/2022b)
      • [Richard] 12.3.0 (upcoming foss/2023a)
      • [Kenneth] 13.1.0
    • OpenBLAS needs easyblock PR for generic build
      • needs a new PR that we can actually merge
    • Rust (to fix build problems with 1.52.1)
    • separate PRs for full 'foss' toolchains
  • next sync meeting
    • Tue 6 June 2023 - 13:00 CEST
Clone this wiki locally