Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIF-165] Write layers directly #42

Merged
merged 5 commits into from
Jun 6, 2024
Merged

[CIF-165] Write layers directly #42

merged 5 commits into from
Jun 6, 2024

Conversation

jterry64
Copy link
Member

What changes were proposed in this PR?

-Add function to write layers directly based on bounding box
-Include option to write out data in smaller tiles, grouped together by a VRT.

How was this patch tested?

Notebook example.

Did this PR include necessary documentation updates?

  • Yes, I am adding a new API feature.

@jterry64 jterry64 requested review from chrowe and S-AI-F May 17, 2024 00:23
@chrowe chrowe requested review from weiqi-tori and removed request for chrowe May 22, 2024 14:22
@@ -52,6 +56,37 @@ def groupby(self, zones, layer=None):
"""
return LayerGroupBy(self.aggregate, zones, layer, self.masks)

def write(self, bbox, output_path, tile_degrees=None):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use MAX_TILE_SIZE as the default tile_degrees?

Write the layer to a path. Does not apply masks.

:param bbox: (min x, min y, max x, max y)
:param output_path: local or s3 path to output to
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output_path needs to be a folder name if written to multiple files, but needs to be a file name with a suffix if written to one file. We could either add instructions for this in the notebook or update the code to make it consistent.

@chrowe
Copy link
Contributor

chrowe commented Jun 1, 2024

I created a new brach to test this with the current code on main https://github.com/wri/cities-cif/tree/feature/write_layers_rebase

It worked to download data with

from city_metrix.layers import TreeCanopyHeight
city_TreeCanopyHeight = TreeCanopyHeight().write(aoi_gdf.total_bounds, 'data/city_TreeCanopyHeight', tile_degrees=0.05)

This produced 100 files totaling 2.75 GB but failed to combine into a vrt. I got 99 of these messages Warning 1: gdalbuildvrt does not support positive NS resolution. Skipping data/city_TreeCanopyHeight/f7c911ec-0721-4ac0-92dc-8bc03eefe610.tif

(Note I first tired with tile_degrees=0.5 and that didn't work, likely because it was trying to download all the data in one file)

@jterry64 jterry64 merged commit db16303 into main Jun 6, 2024
1 check failed
@jterry64 jterry64 deleted the feature/write_layers branch June 6, 2024 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants