You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As we have learned, the retrospective NWM data are stored in a Zarr file in time-major format—that is, it is chunked so that different time slices' data are spatially coherent in storage. We've learned that this ordering of data makes queries on individual river reaches more time-consuming, and I've recently had some issues where the order of operations can result in some very crashy notebooks.
This issue, then, covers the creation of a rechunked version of the full channel routing dataset.
@rajadain@vlulla@elizawallace I'm assuming that there is a task in the proposal that associates with this work item? I can't find one, though, and I'd like to connect this issue to that one. Any ideas? Or did we not have an entry in the proposal for producing a rechunked retro dataset?
We had deferred this task in favor of #104, but it sounds like this should be somewhat quick to do, and if that assessment holds up, then we're probably best to just do this task and retrospectively figure out how much it cost.
As we have learned, the retrospective NWM data are stored in a Zarr file in time-major format—that is, it is chunked so that different time slices' data are spatially coherent in storage. We've learned that this ordering of data makes queries on individual river reaches more time-consuming, and I've recently had some issues where the order of operations can result in some very crashy notebooks.
This issue, then, covers the creation of a rechunked version of the full channel routing dataset.
Connects #104
Closes #26
The text was updated successfully, but these errors were encountered: