Processor layout for 0.25deg configuration

[dougiesquire]

We haven't yet made any consideration of processor layout. Currently, each model component runs sequentially and is allocated a single 48-core node. We need to think about processor layout across the different components, and within each component (e.g. LAYOUT and IO_LAYOUT in MOM).

[aekiss]

Yes, this will take some experimentation and will depend on scalability and load balance within and between components. It will also need to be redone for each resolution.

A first step would be to determine what parameters and diagnostics would be useful to tune and assess load balance.

We should also enable processor land-masking in MOM6 and CICE6 if we haven't already.

CICE has the ability to subdivide the computational domain horizontally into tiles (termed “blocks”), and then parallelise by allocating several blocks to each CPU. This can improve load balancing if a similar number of ice-containing and ice-free blocks are allocated to each CPU. Land-only blocks aren’t allocated to a CPU at all. There are several alternative ways to define the block shape and distribution across processors which can make a big difference to internal load balance within CICE. See Craig et al., (2015) and sec 3.7.2 in the ACCESS-OM2 tech report.

[dougiesquire]

There is some useful info here on choosing processor layout. This includes some info about restrictions imposed by the driver and shows where to find (in CIME) layouts for existing compsets on other machines which may be useful as a starting point.

[micaeljtoliveira]

@dougiesquire Thanks! That's going to be super useful.

[aekiss]

Related: COSIMA/access-om3#125

[aekiss]

Docs on CICE6 block distribution: https://cice-consortium-cice.readthedocs.io/en/cice6.5.0/user_guide/ug_implementation.html#performance

[micaeljtoliveira]

Latest MOM6 version has a new option to automatically perform land block elimination at runtime. This means one does not need to pre-generate a mask table for a given processor count. See here for more details.

[anton-seaice]

This issue is superseded by COSIMA/access-om3#148