Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port EOOffshore CMEMS ASCAT REP and NRT recipes to use beam-refactor branch #456

Open
derekocallaghan opened this issue Dec 22, 2022 · 2 comments

Comments

@derekocallaghan
Copy link
Contributor

Following approach in #445, this will port the CMEMS ASCAT REP and NRT recipes to use the beam-refactor branch. It will involve writing at least one custom PTransform for required Dataset preprocessing.

@derekocallaghan
Copy link
Contributor Author

It looks like two PTransform classes will be required:

  • Similar to the CCMP recipe, the first transform will preprocess each Dataset, including replacing the provided time coordinates.
    • For now, the preprocess function can be wrapped with pangeo_forge_recipes.transforms._add_keys().
  • The second transform will sort the PCollection using the updated time coordinates. I still need to confirm the best approach here, but it's likely that it will update the Position in each Index, something similar to what currently occurs in pangeo_forge_recipes.transforms.IndexItems.

@derekocallaghan
Copy link
Contributor Author

This might be a candidate for the proposed separate issue for migrating tutorials from notebooks -> scripts (with CI testing)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant