Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid using temporary directory if the outdir is local #69

Open
nbren12 opened this issue Mar 24, 2020 · 1 comment
Open

Avoid using temporary directory if the outdir is local #69

nbren12 opened this issue Mar 24, 2020 · 1 comment

Comments

@nbren12
Copy link
Contributor

nbren12 commented Mar 24, 2020

It can be costly to copy every file from the temporary directory to the output directory, so this behavior shouldn't be required for all uses of run_native. After talking to @mcgibbon, any changes here should allow the following uses:

  • want to work in scratch, copy to a persistent or remote location, and delete workdir
  • work in the output location and not delete when done
  • work in a temporary directory and copy to a local or remote location when done
@nbren12
Copy link
Contributor Author

nbren12 commented Mar 25, 2020

TBH, I think this issue is happening because run_native is doing a lot right now. It makes

  1. temporary directories
  2. downloads rundirectories,
  3. runs the fv3model.
  4. uploads/copies the data out of the temporary directory sometimes
  5. capturing logging information.

I think the API is convenient for a certain set of uses, but it makes a lot of assumptions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant