[CMIP6 CMOR-ization & ESGF-publication] NorESM2-MM - historical

[matsbn]

Mandatory information:

Full path to the case(s) of the experiment on NIRD
/projects/NS9560K/noresm/cases
/projects/NS9560K/FRAM/noresm/cases

experiment_id
historical

model_id
NorESM2-MM

CASENAME(s) and years to be CMORized
NHISTfrc2_f09_tn14_20191001, 1850-1949
NHISTfrc2_f09_tn14_20191025, 1950-2014

Optional information

parent_experiment_id
piControl-spinup

parent_experiment_rip
r1i1p1f1

parent_time_units
'days since 0001-01-01'

branch_method
'Hybrid-restart from year 1200-01-01 of piControl-spinup'

other information
The case NHISTfrc2_f09_tn14_20191001 is under /projects/NS9560K/noresm/cases, while case NHISTfrc2_f09_tn14_20191025 is under /projects/NS9560K/FRAM/noresm/cases.

[oyvindseland]

Hi
If there are any way to give this task a priority it would be nice in particular for the last period. Some of the analysis for the NorESM2 coupled simulations use cmorized data.

[YanchunHe]

Years (from the first to the last) needs to rsync to NIRD: /projects/NS9034K/noresm/cases/

2010 2014

[jgriesfeller]

I started working on this, but just selecting the files to be transferred takes very a very long time :-(

[jgriesfeller]

the download to /tos-project1/NS9034K/noresm/cases/NHISTfrc2_f09_tn14_20191025/ has been started. ~2500 files

[jgriesfeller]

The first try stopped after 30 files. :-(

[jgriesfeller]

second try stopped after a similar number of files on login3. This time the 2 other transfer jobs on login1 died as well. Not good

[jgriesfeller]

Tried the same thing by pushing the data from FRAM to NIRD.
Faster, and more stable so far.

[jgriesfeller]

Years 2010 to 2014 have been transferred to /tos-project1/NS9034K/noresm/cases/NHISTfrc2_f09_tn14_20191025
I hope I transferred everything needed to cmorise.
Yanchun, can you start that, please?

[YanchunHe]

It looks the last 4 years was anyhow get cmorized by yesterday...
But it needs to do some final steps before publish.
would you execute under: $CMOR_ROOT/workflow/

./cmorPost.sh -m=NorESM2-MM -e=historical -v=v20191108 &>>logs/cmorPost.log &

After it finishes, check if everything is OK, and update this issue to publish the data (Alok or you will publish).

Please remember, alwalys run checkcmorout.sh in case namelist (/projects/NS9560K/cmor/noresm2cmor/namelists/CMIP6_NorESM2-MM/historical) before doing cmorPost.sh, and remove unfinished 'wrong files' (make sure no running noresm2cmor3 for this experiments).

[jgriesfeller]

cmorPost.sh started

[jgriesfeller]

checkcmorout.sh shows this output, but I don't know how it is supposed to look:
(base) [jang@login-nird-3 historical]$ /projects/NS9560K/cmor/noresm2cmor/namelists/CMIP6_NorESM2-MM/historical/checkcmorout.sh
real: r1i1p1f1
Ofx, fx, etc 8
yyyy1 yyyy2 nf
1849 1859 6
1850 1859 538
1860 1869 544
1870 1879 544
1880 1889 544
1890 1899 544
1900 1909 544
1910 1919 544
1920 1929 544
1930 1939 544
1940 1949 544
1950 1959 608
1960 1969 608
1970 1979 608
1980 1989 608
1990 1999 608
2000 2009 608
2010 2014 608
Total: 9704

Can anybody tell if this is alright or not?

[matsbn]

This seems consistent for the time spans (when adding 1849-1859 and 1850-1859 together) so this should go to ESGF publishing.

[YanchunHe]

Hi Jan,

The number of sha256sum entries is 9133 (/tos-project1/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical/.r1i1p1f1.sha256sum_v20191108),
while ./checkcmorout.sh, shows there are 9704 in total.

And both,
/tos-project1/NS9034K/CMIP6/.cmorout/NorESM2-MM/historical/v20191108.QCreport
/tos-project1/NS9034K/CMIP6/.cmorout/NorESM2-MM/historical/v20191108.links
show total number of checked/linked files are 9704
(v20191108.links have 9705 lines, but the first line is head information).

I don't know the reason, but this usually not happen, and we can usually fix this by relaunch:

/projects/NS9560K/cmor/noresm2cmor/workflow/
./cmorSha256sum.sh -m=NorESM2 -e=historical -v=v20191108 -verbose=false &>>logs/cmorSha256sum.log &

[YanchunHe]

cmorized

data path

• /tos-project1/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical/r1i1p1f1/

version

• v20191108

sha256sum

• .r1i1p1f1.sha256sum_v20191108

(Jan, after you relaunch ./cmorSha256sum.sh, when it finishes, and see if the total entries in the .r1i1p1f1.sha256sum_v20191108 is consistent with other files, then tag a ESGF Publish label).

[YanchunHe]

I have launched the script, and the sha256sum will soon finish.

Jan, you don't need to launch it.

[YanchunHe]

@monsieuralok
sha256sum will soon be ready. Please check if the total number of files in .r1i1p1f1.sha256sum_v20191108 is 9704, it is ready to be published.

[monsieuralok]

@YanchunHe @jgriesfeller Hi, total number of files in /tos-project1/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical/.r1i1p1f1.sha256sum_v20191108 is 5524 while in /tos-project1/NS9034K/CMIP6/.cmorout/NorESM2-MM/historical/v20191108.links 9704 entries. Please could you check it ?

[YanchunHe]

I updated the sha256sum again, and now it is ready to publish.

[YanchunHe]

cmorized burtFractionAll related to issue #286

data path

• /projects/NS9034K/CMIP6/.cmorout/NorESM2-MM/historical/v20210818
• /projects/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical/r1i1p1f1

version

• v20210818

sha256sum
/projects/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical/r1i1p1f1

• .r1i1p1f1.sha256sum_v20210818

[monsieuralok]

@YanchunHe published

[oyvindseland]

Asked for additional variables: 6hrPlevPt variables ps and snowmxrat (1950-2014) and Amon snowmxrat.

[YanchunHe]

cmorized snowmxrat for Emon (#82) and 6hrPlevPt (#109).

data path

• /projects/NS9034K/CMIP6/.cmorout/NorESM2-MM/historical/
• /projects/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical

version

• v20210908

sha256sum
/projects/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical

• .r1i1p1f1.sha256sum_v20210908

[monsieuralok]

@YanchunHe published

[monsieuralok]

@YanchunHe retracted

[YanchunHe]

CMORized additional 3-hourly precipitation datasets.
(only available for 1950-2014)

data path

• /projects/NS9034K/CMIP6/.cmorout/NorESM2-MM/historical/
• /projects/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical

version

• v20230616

sha256sum
/projects/NS9034K/CMIP6/CMIP/NCC/NorESM2-MM/historical

• .r1i1p1f1.sha256sum_v20230616

[monsieuralok]

@YanchunHe published