-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve download stability #19
Comments
There are a few recent additions to the EE API that may make this easier.
|
Just ran some benchmarks on download speed and I don't love the idea of adding a new dependency, but if it can reduce download times substantially and simplify the download system, I think it's worth adding |
With ee.data.computePixels now available in the Python API (as of 2023-02-15), that will probably be the most straightforward way to grab image data. It has the same size limitation as other methods, but allows data to be retrieved directly rather than through an intermediate URL, which should be a win for performance, simplicity, and reliability. Also, this would allow us to avoid adding I need to do some benchmarking to make sure there are no downsides, but at the moment this looks like the way to go. Note that as with all direct |
A quick-and-dirty benchmark test says Time to download a single-band GridMET image at native resolution:
|
The current download system is pretty solid with automated retrying, but the cdsapi package has a more extensive system that should improve download stability. See their implementation for reference.
The text was updated successfully, but these errors were encountered: