-
Notifications
You must be signed in to change notification settings - Fork 6
Home
A short description to ELM on a machine.
-
For MacOS/Linux, see compilers , Libraries, ELM on macOS/linux
-
For Windows/WSL, see ELM on WSL/Ubuntu
-
For a cluster of ORNL, cades, see ELM on cades
Example Cases
E3SM codes are separately into individual components, external submodules, CIME, etc. ELM codes are housed in $E3SM_ROOT/components/elm/src
The Inputs, usually in separate directory, are huge (~12T) for E3SM, even for offline ELM. But offline ELM does actually not require all of them. For one point (grid) offline ELM, the input data is about ~800M, but usually what data required is unknown to users.
Example point data (full package for a few sites): [NGEE-Arctic point ELM input] (https://github.com/fmyuan/pt-e3sm-inputdata)
THREE (3) datasets: I) domain (mesh): under share/domains/domain.clm; II) (atm.)forcing: under atm/datm7/; III) land surface properties (inc. parameters): under lnd/clm2/, including- './paramdata/' for parameter files, './surfdata_map' for surface property files, ...
E3SM contains a huge amount of Scripts & Settings to set up, build and run model(s), and/or to process inputs/outputs. Scripts in various languages, including python, shell, perl, xml, ...
Most important Scripts/Settings: './cime', './cime_config' (for machine settings, and component building).
For an application, user will create a 'CASE', which usually under pre-defined or user-defined directory (CASE_ROOT).
For an established CASE, model usually runs in 2 or 3 stages.
Pre-1850 SPINUP, for obtaining an equilibrium state under pre-defined forcing period and fixed surface properties. During this Pre-1850 SPINUP, it's needed in some cases to do an accelerated SPINUP ('ad_spinup') for speeding up some very slow processes, e.g. slowly decomposed Soil Organic Matters.
Then run CASE from 1850-2014 or so (ending time upon forcing data), called Transient run, under dynamical land uses and land changes, rising CO2, warming, aersols (including N deposition), etc.
If forcing and/or other land property dynamics available from Scenario model and analysis, then run Cases as 'scenario run'.
NOTE: The whole runs are continuous, even under different forcings/properties. It implies that later run stage is using RESTART state files, generated from pre-stage.
OPTIONAL if model has already ported into a machine. And model is in that machine. OTHERWISE, do following to get model from repository:
git clone https://github.com/E3SM-Project/E3SM.git ./E3SM
cd E3SM
git submodule init
git submodule update
CHECK $E3SM_ROOT/cime_config/config_machines.xml, for supported machines. CHECK $E3SM_ROOT/cime_config/config_compilers.xml, for supported compilers in specified machines.
IF NOT there, have to port model by users. What to do IS to set up machine environments (linux OS usually), compilers, data directory, computing environments and resources, and so on. JUST follow already-ported machines/compilers similar to user's.
TIPS: Beside general LINUX OS and default tools/utilities, what shall be included for E3SM: Python, bash shell, csh shell, perl5 (with module XML::LibXML, depending on libxml2 and xmllint), cmake. ADDITIONALLY must have: c/c++/fortran compilers (threading support preferred), mpich/openmpi (optional but preferred), netcdf (netcdf4 parallel preferred), BLAS/LAPACK.
VERY CRITICAL: compiers, mpi, and netcdf, all MUST be built consistently (compatibly)! This is specially critical to multiple versions of softwares/tools in a cluster or supper computer, even a laptop/desktop with multiple compilers in various directories (including such as anacoda python env.).
cd $E3SM_ROOT/cime/scripts
./create_newcase --case $CASE_ROOT/TestELM_ICB1850CNPRDCTCBC --mach $MACHINE_NAME --compset ICB1850CNPRDCTCBC --res ELM_USRDAT --mpilib $MPINAME --walltime 6:00:00 --handle-preexisting-dirs u --compiler $COMPILER_NAME
NOTE: $XXXXX are your choices, but must be existing in your system and defined in both 'cime_config/machines/config_machines.xml', and, 'config_compilers.xml'. And also, all other default settings are pre-defined in those 2 *.xml files, and OPTIONALLY 2 or 3 more *.xml, 'config_pio.xml', 'config_batch.xml', 'config_workflow.xml', relevant to parallel I/O and/or computing resources.
This command would create 3 directories, CASE_DIRECTORY (e.g. here $CASE_ROOT/TestELM_ICB1850CNPRDCTCBC), CASE_BUILD_DIRECTORY, CASE_RUN_DIRECTORY, the latter 2 of which are usually under: $RUN_ROOT/$CASE_NAME/bld, $RUN_ROOT/$CASE_NAME/run, for e3sm.exe building and running.
TIPS: Under cime/scripts, there are 2 more 'create_' and 2 'query_' python scripts, try run as following to know how to use it, e.g.
./create_newcase --help
GO to $CASE_DIRECTORY:
cd $CASE_ROOT/TestELM_ICB1850CNPRDCTCBC
SET UP case, by editing a few *.xml files: 'env_case.xml', 'env_build.xml', 'env_run.xml', and optionally 'env_mach_specific.xml', 'env_mach_pes.xml', and then,
./case_setup
IF ANY ERROR, try to edit those *.xml files to configure case as in IV.
Using any xml editor to edit configurations, OR, try run
./xmlchange $SETTING_ID=VAL or ./xmlchange --id $SETTING_ID --val VAL
$SETTING_ID is any variable ID to set in *.xml. THERE ARE a few ID to change in the case created here, for examples,
PIO_VERSION ('env_build.xml'), ELM_USRDAT_NAME, ELM_ACCELERATED_SPINUP, ELM_FORCE_COLDSTART, PIO_TYPENAME, ... ('env_run.xml')
There are 2 or 3 'user_nl_*' files, which can be edited to over-ride settings in those *.xml files.
OPTIONALLY, edit 'env_mach_specific.xml' for system compilers/env. settings, and 'Macro.cmake' or 'Macro.make' for building cmake/make options. THEN
./case.build
The model building logs would be in $CADE_BUILD_DIRECTORY. If ANY ERROR, please check those logs.
FOR RUNNING a case,
- editing 'env_run.xml' for the following settings:
STOP_N, STOP_OPITONS, REST_N, REST_OPTIONS, for running time period (length and unit), and frequency of restart file output.
- IF KNOW how to, customizing outputs, by editing 'user_nl_elm' for:
hist_empty_htapes
hist_fincl1, and/or add hist_fincl2, ...
hist_dov2xy
hist_nhtfrq
hist_mfilt
hist_avgflag_pertape
- THEN,
./case.submit
AND, results would be in $CASE_RUN_DIRECTORY, with logs files.
FOR any reason, model pauses or breaks, e.g. due to time limit, user need to resume the model run, by editing 'env_run.xml' as,
./xmlchange CONTINUE_RUN=TRUE
and,
./case.submit
TIP: Create, Setup, Configure, Build, and Run multi-stage offline ELM in ONE SINGLE STEP, using OLMT. Please check out the example: CASE IV
UPDATED (2018-09-25): GCC 8.2 installation for High Sierra (darwin 10.13.6)
A Briefing Guide on CADES@ornl
See PFLOTRAN-ELM interface README Or, PFLOTRAN-ELM interface source code INSTRUCTIONS Or, PFLOTRAN-ELM interface wiki