Add case3_MARRMOT_calibration_ensemble_RunSavanah notebook

[SarahAlidoost]

To create ewatercycle conda environment, see eWaterCycle/ewatercycle#234

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[sverhoeven]

I submitted the nb on cartesius.
After creation of ewatercycle conda env I also had to install cma and jupyter.
I also had to copy ~/.config/ewatercycle/ewatercycle.yaml .

Currently seeing load of 13 (out of 24), so I think the popsize can be increased to 18 if we want to.

[sverhoeven]

Job completed OK

CPU Utilized: 1-14:50:34
CPU Efficiency: 45.960f 3-12:30:48 core-walltime
Job Wall-clock time: 03:31:17

[sverhoeven]

Great, I got the same result. I like the amount of docs.

See nbreview comments for questions/suggestions.

[SarahAlidoost]

Great, I got the same result. I like the amount of docs.

See nbreview comments for questions/suggestions.

Many thanks for reviewing, your comments are addressed. Please have another look.

[sverhoeven]

I submitted the notebook again to check the log messages.

[sverhoeven]

Hmm, got error

PermissionError                           Traceback (most recent call last)
/scratch-local/verhoes.10667541/ipykernel_8631/1173285456.py in <module>
      2 filename = f"{CFG['output_dir']}/marrmot-m14_ERA5_Savannah_calibration_{CALIBRATION['start']}_{CALIBRATION['end']}.yml"
      3 filename = "/scratch/shared/ewatercycle/case3_marrmot14/marrmot-m14_ERA5_Savannah_calibration_1991-01-01T00:00:00Z_2001-12-31T00:00:00Z.yml"
----> 4 with open(filename) as f:
      5     calibration_results = yaml.load(f)
      6 fig = plot_parameters(calibration_results)

PermissionError: [Errno 13] Permission denied: '/scratch/shared/ewatercycle/case3_marrmot14/marrmot-m14_ERA5_Savannah_calibration_1991-01-01T00:00:00Z_2001-12-31T00:00:00Z.yml'

Did you remove /scratch/shared/ewatercycle/case3_marrmot14?

[SarahAlidoost]

Hmm, got error

PermissionError                           Traceback (most recent call last)
/scratch-local/verhoes.10667541/ipykernel_8631/1173285456.py in <module>
      2 filename = f"{CFG['output_dir']}/marrmot-m14_ERA5_Savannah_calibration_{CALIBRATION['start']}_{CALIBRATION['end']}.yml"
      3 filename = "/scratch/shared/ewatercycle/case3_marrmot14/marrmot-m14_ERA5_Savannah_calibration_1991-01-01T00:00:00Z_2001-12-31T00:00:00Z.yml"
----> 4 with open(filename) as f:
      5     calibration_results = yaml.load(f)
      6 fig = plot_parameters(calibration_results)

PermissionError: [Errno 13] Permission denied: '/scratch/shared/ewatercycle/case3_marrmot14/marrmot-m14_ERA5_Savannah_calibration_1991-01-01T00:00:00Z_2001-12-31T00:00:00Z.yml'

Did you remove /scratch/shared/ewatercycle/case3_marrmot14?

No, the group permission wasn't set. Now it should work.

[sverhoeven]

Job completed OK.

CPU Utilized: 1-14:50:22
CPU Efficiency: 45.820f 3-12:46:24 core-walltime
Job Wall-clock time: 03:31:56
Memory Utilized: 2.66 GB
Memory Efficiency: 4.420f 60.31 GB

The ESMValToolDeprecationWarning are not being silenced because ewatercycle import is done before filterwarnings. Can you move logging/warnings block above the imports?

Also the container log messages still appear. I think its because message is printed using root logger see eWaterCycle/grpc4bmi#103 . To silence the container message I had to set the global level to WARNING with

import logging
logging.basicConfig(level=logging.WARNING)

Can you replace existing logging code with this?

[SarahAlidoost]

Job completed OK.
The ESMValToolDeprecationWarning are not being silenced because ewatercycle import is done before filterwarnings. Can you move logging/warnings block above the imports?

Also the container log messages still appear. I think its because message is printed using root logger see eWaterCycle/grpc4bmi#103 . To silence the container message I had to set the global level to WARNING with

import logging
logging.basicConfig(level=logging.WARNING)

Can you replace existing logging code with this?

Thanks @sverhoeven. I fixed the loggers. Also, I added two variables "POPSIZE" and "MAXITER", please see my last commit.

[sverhoeven]

Thanks for exposing POPSIZE and MAXITER,
I submitted nb with POPSIZE=18 and commented out hardcoded filename in last cell.

[sverhoeven]

Output cells look very silent now.

I copied my completed notebook, slurm log, output fles to /scratch/shared/ewatercycle/case3_marrmot14/byverhoes/

[SarahAlidoost]

Thanks for exposing POPSIZE and MAXITER,
I submitted nb with POPSIZE=18 and commented out hardcoded filename in last cell.

Many thanks for reviewing and testing the notebook.