including regridding in diagnostics code vs. preprocessing

[clittleaer]

Hi all -- general question about the MDTF philosophy on regridding model/obs datasets within the diagnostic code or outside it...

My current approach to designing sea level diagnostics is to regrid (fine) observational datasets to (generally coarser, but often more complex) ocean model grids using xESMF tools. This can be a little computationally-intensive, and involves a specification of the ocean grid (h+q points). I'm not sure: 1) whether this process belongs in the diagnostic code (the alternative is to preprocess the obs datasets for each model grid) and 2) where the ocean model grid specification file should be kept/brought in if I include the regridding as part of the diagnostic. Are there any established strategies, or thoughts along these lines?

Thanks!

[wrongkindofdoctor]

@clittleaer I'd recommend packaging the regridding code with your POD--either as a separate tool the user calls on the desired gridspec and raw data before running the framework, or a module that can be called from the driver script via a user environment variable (pod_env_vars)specified in the settings.jsonc file (e.g. "regrid": "TRUE"). The former has the advantage of not having to switch the pod_env_var between TRUE/FALSE, and would be easy to update to call from the main workflow in the next framework. The latter might be somewhat simpler to do since you won't have to make a separate CLI, but relies on the user remembering to switch the pod_env_var on/off. Either way, Include instructions for regridding the data, and cite an example gridspec file in the POD README (e.g., the name, typical location, and description of the required variables).

The vision for the next iteration of the framework is to allow users to specify custom processing scripts in the runtime configuration file, with the potential to add tools required by a majority of the supported PODs to the framework itself. For now, though, non-standard preprocessing needs to occur at the POD level.

As for the gridspec file itself, it is best not to store binary files with the code base, especially since PODs are meant to run with any supported dataset. We could look into hosting the sample data locally and on the NCAR FTP server, but otherwise, it is up to the user to obtain the necessary files.

[clittleaer]

Thanks for the suggestions @wrongkindofdoctor. I'll consider which one will work best over the next few weeks. Maybe more questions to follow.

[afunktamu]

@wrongkindofdoctor Hi all - I'm new here, but wanted to inquire if there was a known completion date for the next version of the framework that was mentioned above.