Additional 3 GH200 + module updates

.github/workflows/ci.yml

@@ -33,7 +33,7 @@ jobs:
         run: sphinx-build -W . ./html
 
       - name: Archive built docs
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: docs
           path: html

conf.py

@@ -94,7 +94,7 @@
     # Code highlighting theme for dark mode
     "pygment_dark_style": "github-dark-high-contrast",
     # Add an announcement bar, visible at the top of each page.
-    "announcement": "3 NVIDIA Grace-Hopper nodes (GH200 480) are now available. See Using Bede for more information.",
+    "announcement": "3 additional (6 total) NVIDIA Grace-Hopper nodes (GH200 480) are now available. See Using Bede",
     # Add the traditional footer theme and sphinx acknowledgements
     "extra_footer": f"<p>&nbsp;Built with <a href=\"http://sphinx-doc.org/\">Sphinx</a> {sphinx.__version__} using a theme by the <a href=\"https://ebp.jupyterbook.org/\">Executable Book Project</a>.</p>"
 }

guides/nvidia-profiling-tools.rst

@@ -59,10 +59,7 @@ Once this file has been downloaded to your local machine, it can be opened in ``
 Cluster Modules
 ~~~~~~~~~~~~~~~
 
-``nsys`` is available through the following Bede modules:
-
-* ``nsight-systems/2020.3.1``
-* ``nvhpc/20.9``
+Bede modules which provide ``nsys`` and which version can be found on :ref:`the Nsight Systems page<software-tools-nsight-systems>`.
 
 More Information
 ~~~~~~~~~~~~~~~~
@@ -105,10 +102,7 @@ Once the ``.ncu-rep`` file has been downloaded locally, it can be imported into
 Cluster Modules
 ~~~~~~~~~~~~~~~
 
-``ncu`` is available through the following Bede modules:
-
-* ``nsight-compute/2020.2.1``
-* ``nvhpc/20.9``
+Bede modules which provide ``ncu`` and which version can be found on :ref:`the Nsight Compute page <software-tools-nsight-compute>`.
 
 
 More Information

hardware/index.rst

@@ -53,7 +53,7 @@ There are:
    - 480 GB LPDDR5X RAM
    - 1x Mellanox CONNECTX-7 NDR200 (100Gb/s due to existing network) InfiniBand port
 
--  2x ``gh`` nodes, each containing
+-  5x ``gh`` nodes, each containing
 
    - 1x `NVIDIA Grace Hopper Superchip <https://www.nvidia.com/en-gb/data-center/grace-hopper-superchip/>`_ (GH200 480GB)
 

software/compilers/gcc.rst

@@ -18,6 +18,7 @@ offload support:
 
    .. code-tab:: bash aarch64
 
+      module load gcc/14.2
       module load gcc/13.2
       module load gcc/12.2
 

software/compilers/nvcc.rst

@@ -13,6 +13,7 @@ Unlike other compiler modules, the cuda modules do not set ``CC`` or ``CXX`` env
 
       module load cuda
 
+      module load cuda/12.4.1
       module load cuda/12.0.1
       module load cuda/11.5.1
       module load cuda/11.4.1
@@ -25,6 +26,9 @@ Unlike other compiler modules, the cuda modules do not set ``CC`` or ``CXX`` env
 
       module load cuda
 
+      module load cuda/12.6.1
+      module load cuda/12.5.1
+      module load cuda/12.4.1
       module load cuda/12.3.2
       module load cuda/12.2.2
       module load cuda/12.1.1

software/compilers/nvhpc.rst

@@ -7,7 +7,7 @@ The `NVIDIA HPC SDK <https://developer.nvidia.com/hpc-sdk>`__, otherwise referre
 It provides C, C++ and Fortran compilers, which include features enabling GPU acceleration through standard C++ and Fortran, OpenACC directives and CUDA.
 
 It is provided for use on the system by the ``nvhpc`` module(s).
-It provides the ``nvc``, ``nvc++`` and ``nvfortran`` compilers.
+It provides the ``nvc``, ``nvc++`` and ``nvfortran`` compilers, and includes atleast one copy of the CUDA toolkit (including ``nvcc``).
 
 This module also provides the `NCCL <https://docs.nvidia.com/deeplearning/nccl/user-guide/docs/index.html>`__ and `NVSHMEM <https://docs.nvidia.com/hpc-sdk/nvshmem/index.html>`__ libraries, as well as the suite of math libraries typically included with the CUDA Toolkit, such as ``cublas``, ``cufft`` and ``nvblas``.
 
@@ -25,6 +25,7 @@ This module also provides the `NCCL <https://docs.nvidia.com/deeplearning/nccl/u
 
       module load nvhpc
 
+      module load nvhpc/24.9
       module load nvhpc/24.1
 
 For further information please see the `NVIDIA HPC SDK Documentation Archive <https://docs.nvidia.com/hpc-sdk/archive/>`__.

software/tools/nsight-compute.rst

@@ -19,6 +19,7 @@ You should use a versions of ``ncu`` that is at least as new as the CUDA toolkit
       module load nsight-compute/2022.1.0
       module load nsight-compute/2020.2.1
 
+      module load cuda/12.4.1 # provides ncu 2024.1.1
       module load cuda/12.0.1 # provides ncu 2022.4.1
       module load cuda/11.5.1 # provides ncu 2021.3.1
       module load cuda/11.4.1 # provides ncu 2021.2.1
@@ -33,13 +34,17 @@ You should use a versions of ``ncu`` that is at least as new as the CUDA toolkit
 
       module load nsight-systems/2023.4.1
 
+      module load cuda/12.6.1 # provides ncu 2024.3.2
+      module load cuda/12.5.1 # provides ncu 2024.2.1
+      module load cuda/12.4.1 # provides ncu 2024.1.1
       module load cuda/12.3.2 # provides ncu 2023.3.1
       module load cuda/12.2.2 # provides ncu 2023.2.2
       module load cuda/12.1.1 # provides ncu 2023.1.1
       module load cuda/11.8.0 # provides ncu 2022.3.0
       module load cuda/11.7.1 # provides ncu 2022.1.0
       module load cuda/11.7.0 # provides ncu 2022.2.0
 
+      module load nvhpc/24.9  # provides ncu 2024.3.0
       module load nvhpc/24.1  # provides ncu 2023.3.1
 
 

software/tools/nsight-systems.rst

@@ -20,6 +20,7 @@ You should use a versions of ``nsys`` that is at least as new as the CUDA toolki
       module load nsight-systems/2022.1.1
       module load nsight-systems/2020.3.1
 
+      module load cuda/12.4.1 # provides nsys 2023.4.4
       module load cuda/12.0.1 # provides nsys 2022.4.2
       module load cuda/11.5.1 # provides nsys 2021.3.3
       module load cuda/11.4.1 # provides nsys 2021.2.4
@@ -34,13 +35,17 @@ You should use a versions of ``nsys`` that is at least as new as the CUDA toolki
 
       module load nsight-systems/2023.4.1
 
+      module load cuda/12.6.1 # provides nsys 2024.5.1
+      module load cuda/12.5.1 # provides nsys 2024.2.3
+      module load cuda/12.4.1 # provides nsys 2023.4.4
       module load cuda/12.3.2 # provides nsys 2023.3.3
       module load cuda/12.2.2 # provides nsys 2023.2.3
       module load cuda/12.1.1 # provides nsys 2023.1.2
       module load cuda/11.8.0 # provides nsys 2022.4.2
       module load cuda/11.7.1 # provides nsys 2022.1.3
       module load cuda/11.7.0 # provides nsys 2022.1.3
 
+      module load nvhpc/24.9  # provides nsys 2024.5.1
       module load nvhpc/24.1  # provides nsys 2023.4.1
 
 To generate an application timeline with Nsight Systems CLI (``nsys``):

usage/index.rst

@@ -470,7 +470,7 @@ Jobs are scheduled subject to Slurm's `Multifactor Priority Plugin <https://slur
 Grace-Hopper Pilot
 ------------------
 
-Bede contains 3 NVIDIA Grace-Hopper nodes.
+Bede contains 6 NVIDIA Grace-Hopper nodes.
 
 Each Grace-Hopper node contains a single `Grace Hopper Superchip <https://www.nvidia.com/en-gb/data-center/grace-hopper-superchip/>`_, containing one 72-core 64-bit ARM CPU and one 96GB Hopper GPU with NVLink-C2C providing 900GB/s of bidirectional bandwidth between the CPU and GPU. Further details are listed on the :ref:`hardware` page.
 
@@ -521,7 +521,7 @@ Key differences to be aware of include:
 
 * :ref:`PyTorch<software-applications-pytorch>`
 
-  * Current (at least up to ``2.1.0``) builds of pytorch provided via conda or pip for ``aarch64`` do not include cuda support (``torch.cuda.is_available()`` returns ``false``).
+  * Installation options for pre-built pytorch for ``aarch64`` platforms with CUDA support enabled is limited. As of July 2024, only ``pip`` installs of pytorch ``2.4.0`` or newer for ``CUDA 12.4`` are suitable. See :ref:`software-applications-pytorch` for more information.
   * NVIDIA provide `NGC Pytorch containers <https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch/tags>`_ which can be used instead, with pytorch installed into the default python environment.
 
     * These containers are large, so you will likely want to set your ``APPTAINER_CACHEDIR`` environment variable to avoid filling your home directory quota