Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create JEDI class #2805

Merged
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
72cfc5b
Initial commit
DavidNew-NOAA Aug 2, 2024
dddaec2
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Aug 5, 2024
2a313b2
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Aug 8, 2024
b2536c5
Move YAML save out of get_jedi_config method
DavidNew-NOAA Aug 8, 2024
a6bc7cc
pynorms
DavidNew-NOAA Aug 8, 2024
f5f4d94
Update exglobal_atmens_analysis_letkf.py
DavidNew-NOAA Aug 8, 2024
e5b7409
Addressing Rahul's comments
DavidNew-NOAA Aug 21, 2024
0ff0521
Merge branch 'feature/jedi_class' of https://github.com/DavidNew-NOAA…
DavidNew-NOAA Aug 21, 2024
e0bd76c
Move jedi.py to its own directory and move logging from jedi.py funct…
DavidNew-NOAA Aug 21, 2024
6617f8b
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Aug 21, 2024
e85e993
Finished debugging
DavidNew-NOAA Aug 21, 2024
43a6338
pynorms
DavidNew-NOAA Aug 21, 2024
ef68fad
Update some comments and logging
DavidNew-NOAA Aug 22, 2024
0d01ad0
pynorms
DavidNew-NOAA Aug 22, 2024
d496307
pynorms
DavidNew-NOAA Aug 22, 2024
6b95386
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Aug 28, 2024
056d07c
Merge latest develop
DavidNew-NOAA Aug 29, 2024
44f172c
Refactor LETKF observer to work with JEDI class
DavidNew-NOAA Aug 29, 2024
55efedf
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Aug 29, 2024
e06f736
Fix small bug
DavidNew-NOAA Aug 29, 2024
4da3d34
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Aug 29, 2024
3c6d5b5
Redo some things w/ JEDI class to address YAML archiving error
DavidNew-NOAA Aug 30, 2024
79aca48
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Aug 30, 2024
e0e3f40
pynorms
DavidNew-NOAA Aug 30, 2024
a6f4b81
Proposed change to enkf.yaml.j2
DavidNew-NOAA Aug 30, 2024
ed2613b
Revert "Proposed change to enkf.yaml.j2"
DavidNew-NOAA Aug 30, 2024
86b604a
Fix error from my misunderstanding of how obs/sol jobs work for LETKF
DavidNew-NOAA Aug 30, 2024
9b70195
Set JCB algorithm depending on whether running in observer-solver mod…
DavidNew-NOAA Sep 3, 2024
c279e45
Fix error in JCB prototype file name
DavidNew-NOAA Sep 3, 2024
1e034f7
Address some comments
DavidNew-NOAA Sep 3, 2024
526b961
The perils of copypasta
DavidNew-NOAA Sep 3, 2024
d3ecab3
Address comments and update GDAS hash
DavidNew-NOAA Sep 4, 2024
02b3734
Mistakenly had class method marked as static method
DavidNew-NOAA Sep 4, 2024
3d3ccf0
Update
DavidNew-NOAA Sep 4, 2024
d5e2fe0
Update jedi.py
DavidNew-NOAA Sep 4, 2024
ea97ddb
Merge branch 'develop' into feature/jedi_class
DavidNew-NOAA Sep 4, 2024
16868a8
Update jedi.py
DavidNew-NOAA Sep 4, 2024
a423c6f
Update ush/python/pygfs/jedi/jedi.py
DavidNew-NOAA Sep 5, 2024
35c3f49
Update ush/python/pygfs/jedi/jedi.py
DavidNew-NOAA Sep 5, 2024
9a24b48
Update ush/python/pygfs/jedi/jedi.py
DavidNew-NOAA Sep 5, 2024
f1cb728
Update ush/python/pygfs/jedi/jedi.py
DavidNew-NOAA Sep 5, 2024
3bb5336
Update ush/python/pygfs/task/atm_analysis.py
DavidNew-NOAA Sep 5, 2024
ba19cdf
Update ush/python/pygfs/task/atmens_analysis.py
DavidNew-NOAA Sep 5, 2024
287a949
Update ush/python/pygfs/task/atm_analysis.py
DavidNew-NOAA Sep 5, 2024
934d036
Update ush/python/pygfs/task/atm_analysis.py
DavidNew-NOAA Sep 5, 2024
806010f
Update ush/python/pygfs/task/atm_analysis.py
DavidNew-NOAA Sep 5, 2024
79c4d60
Address final comments
DavidNew-NOAA Sep 5, 2024
50db563
Merge branch 'feature/jedi_class' of https://github.com/DavidNew-NOAA…
DavidNew-NOAA Sep 5, 2024
00874e9
Forgot some comments
DavidNew-NOAA Sep 5, 2024
c039881
Final comment block
DavidNew-NOAA Sep 5, 2024
fc437a8
pynorms
DavidNew-NOAA Sep 5, 2024
95974c3
Small bug
DavidNew-NOAA Sep 5, 2024
bc52fcb
Update ush/python/pygfs/task/atmens_analysis.py
aerorahul Sep 5, 2024
f445e28
Update ush/python/pygfs/task/atm_analysis.py
aerorahul Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions parm/archive/enkf.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,14 @@ enkf:
"radstat.ensmean"] %}
{% else %}
{% if lobsdiag_forenkf %}
{% set da_files = ["atmens_observer.yaml",
"atmens_solver.yaml",
{% set da_files = ["atmensanlobs.yaml",
"atmensanlsol.yaml",
"atmensanlfv3inc.yaml",
"atminc.ensmean.nc",
"atmensstat"] %}
{% else %}
{% set da_files = ["atmens.yaml",
{% set da_files = ["atmensanlletkf.yaml",
"atmensanlfv3inc.yaml",
"atminc.ensmean.nc",
"atmensstat"] %}
{% endif %}
Expand Down
3 changes: 2 additions & 1 deletion parm/archive/gdas.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,8 @@ gdas:

# Analysis state
{% if DO_JEDIATMVAR %}
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmvar.yaml"
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmanlvar.yaml"
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmanlfv3inc.yaml"
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmstat"
{% else %}
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}gsistat"
Expand Down
3 changes: 2 additions & 1 deletion parm/archive/gfsa.yaml.j2
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@ gfsa:

# State data
{% if DO_JEDIATMVAR %}
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmvar.yaml"
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmanlvar.yaml"
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmanlfv3inc.yaml"
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}atmstat"
{% else %}
- "{{ COMIN_ATMOS_ANALYSIS | relpath(ROTDIR) }}/{{ head }}gsistat"
Expand Down
6 changes: 5 additions & 1 deletion parm/config/gfs/config.atmensanl
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@
echo "BEGIN: config.atmensanl"

export JCB_BASE_YAML="${PARMgfs}/gdas/atm/jcb-base.yaml.j2"
export JCB_ALGO_YAML=@JCB_ALGO_YAML@
if [[ ${lobsdiag_forenkf} = ".false." ]] ; then
export JCB_ALGO_YAML=@JCB_ALGO_YAML_LETKF@
else
export JCB_ALGO_YAML=@JCB_ALGO_YAML_OBS@
fi

export INTERP_METHOD='barycentric'

Expand Down
3 changes: 2 additions & 1 deletion parm/config/gfs/yaml/defaults.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ atmanl:
IO_LAYOUT_Y: 1

atmensanl:
JCB_ALGO_YAML: "${PARMgfs}/gdas/atm/jcb-prototype_lgetkf.yaml.j2"
JCB_ALGO_YAML_LETKF: "${PARMgfs}/gdas/atm/jcb-prototype_lgetkf.yaml.j2"
JCB_ALGO_YAML_OBS: "${PARMgfs}/gdas/atm/jcb-prototype_lgetkf_observer.yaml.j2"
LAYOUT_X_ATMENSANL: 8
LAYOUT_Y_ATMENSANL: 8
IO_LAYOUT_X: 1
Expand Down
2 changes: 2 additions & 0 deletions scripts/exglobal_atm_analysis_finalize.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,6 @@

# Instantiate the atm analysis task
AtmAnl = AtmAnalysis(config)

# Finalize JEDI variational analysis
AtmAnl.finalize()
12 changes: 7 additions & 5 deletions scripts/exglobal_atm_analysis_fv3_increment.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# exglobal_atm_analysis_fv3_increment.py
# This script creates an AtmAnalysis object
# and runs the init_fv3_increment and fv3_increment methods
# and runs the initialize_fv3inc and execute methods
# which convert the JEDI increment into an FV3 increment
import os

Expand All @@ -17,7 +17,9 @@
# Take configuration from environment and cast it as python dictionary
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atm analysis task
AtmAnl = AtmAnalysis(config)
AtmAnl.init_fv3_increment()
AtmAnl.fv3_increment()
# Instantiate the atm analysis object
AtmAnl = AtmAnalysis(config, 'atmanlfv3inc')

# Initialize and execute FV3 increment converter
AtmAnl.initialize_jedi()
AtmAnl.execute(config.APRUN_ATMANLFV3INC)
7 changes: 5 additions & 2 deletions scripts/exglobal_atm_analysis_initialize.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,8 @@
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atm analysis task
AtmAnl = AtmAnalysis(config)
AtmAnl.initialize()
AtmAnl = AtmAnalysis(config, 'atmanlvar')

# Initialize JEDI variational analysis
AtmAnl.initialize_jedi()
AtmAnl.initialize_analysis()
6 changes: 4 additions & 2 deletions scripts/exglobal_atm_analysis_variational.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,7 @@
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atm analysis task
AtmAnl = AtmAnalysis(config)
AtmAnl.variational()
AtmAnl = AtmAnalysis(config, 'atmanlvar')

# Execute JEDI variational analysis
AtmAnl.execute(config.APRUN_ATMANLVAR, ['fv3jedi', 'variational'])
2 changes: 2 additions & 0 deletions scripts/exglobal_atmens_analysis_finalize.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,6 @@

# Instantiate the atmens analysis task
AtmEnsAnl = AtmEnsAnalysis(config)

# Finalize ensemble DA analysis
AtmEnsAnl.finalize()
12 changes: 7 additions & 5 deletions scripts/exglobal_atmens_analysis_fv3_increment.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
# exglobal_atmens_analysis_fv3_increment.py
# This script creates an AtmEnsAnalysis object
# and runs the init_fv3_increment and fv3_increment methods
# and runs the initialize_fv3inc and execute methods
# which convert the JEDI increment into an FV3 increment
import os

Expand All @@ -17,7 +17,9 @@
# Take configuration from environment and cast it as python dictionary
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atmens analysis task
AtmEnsAnl = AtmEnsAnalysis(config)
AtmEnsAnl.init_fv3_increment()
AtmEnsAnl.fv3_increment()
# Instantiate the atmens analysis object
AtmEnsAnl = AtmEnsAnalysis(config, 'atmensanlfv3inc')

# Initialize and execute JEDI FV3 increment converter
AtmEnsAnl.initialize_jedi()
AtmEnsAnl.execute(config.APRUN_ATMENSANLFV3INC)
10 changes: 8 additions & 2 deletions scripts/exglobal_atmens_analysis_initialize.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,11 @@
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atmens analysis task
AtmEnsAnl = AtmEnsAnalysis(config)
AtmEnsAnl.initialize()
if not config.lobsdiag_forenkf:
AtmEnsAnl = AtmEnsAnalysis(config, 'atmensanlletkf')
else:
AtmEnsAnl = AtmEnsAnalysis(config, 'atmensanlobs')

# Initialize JEDI ensemble DA analysis
AtmEnsAnl.initialize_jedi()
AtmEnsAnl.initialize_analysis()
10 changes: 6 additions & 4 deletions scripts/exglobal_atmens_analysis_letkf.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
# exglobal_atmens_analysis_letkf.py
# This script creates an AtmEnsAnalysis object
# and runs the letkf method
# which executes the global atm local ensemble analysis
# and runs the execute method which executes
# the global atm local ensemble analysis
import os

from wxflow import Logger, cast_strdict_as_dtypedict
Expand All @@ -18,5 +18,7 @@
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atmens analysis task
AtmEnsAnl = AtmEnsAnalysis(config)
AtmEnsAnl.letkf()
AtmEnsAnl = AtmEnsAnalysis(config, 'atmensanlletkf')

# Execute the JEDI ensemble DA analysis
AtmEnsAnl.execute(config.APRUN_ATMENSANLLETKF, ['fv3jedi', 'localensembleda'])
7 changes: 4 additions & 3 deletions scripts/exglobal_atmens_analysis_obs.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atmens analysis task
AtmEnsAnl = AtmEnsAnalysis(config)
AtmEnsAnl.init_observer()
AtmEnsAnl.observe()
AtmEnsAnl = AtmEnsAnalysis(config, 'atmensanlobs')

# Initialize and execute JEDI ensembler DA analysis in observer mode
AtmEnsAnl.execute(config.APRUN_ATMENSANLOBS, ['fv3jedi', 'localensembleda'])
8 changes: 5 additions & 3 deletions scripts/exglobal_atmens_analysis_sol.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
config = cast_strdict_as_dtypedict(os.environ)

# Instantiate the atmens analysis task
AtmEnsAnl = AtmEnsAnalysis(config)
AtmEnsAnl.init_solver()
AtmEnsAnl.solve()
AtmEnsAnl = AtmEnsAnalysis(config, 'atmensanlsol')

# Initialize and execute JEDI ensemble DA analysis in solver mode
AtmEnsAnl.initialize_jedi()
AtmEnsAnl.execute(config.APRUN_ATMENSANLSOL, ['fv3jedi', 'localensembleda'])
Empty file.
Loading
Loading