Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The definition of maxRelativeDepth of Dggrs #80

Open
tik65536 opened this issue Dec 7, 2024 · 8 comments
Open

The definition of maxRelativeDepth of Dggrs #80

tik65536 opened this issue Dec 7, 2024 · 8 comments

Comments

@tik65536
Copy link

tik65536 commented Dec 7, 2024

The DGGRS definition included both "maxRefinementLevel" and "maxRelativeDepth", which from the description of the yaml file, they are similar. Both define the max refinement level in an absolute or relative sense. But we can always check if the request zone_level + relative_level >= maxRefinementLevel, without the use maxRelativeDepth. So, what's the actual usage of maxRelativeDepth ?

Another interpretation from the API implementation perspective is to limit the number of level returns in one API call. But that is not Dggrs definition related.

Thx,
Tik

@tik65536 tik65536 changed the title The definition of maxRelativeDepth of Dggrs definition The definition of maxRelativeDepth of Dggrs Dec 7, 2024
@jerstlouis
Copy link
Member

jerstlouis commented Dec 7, 2024

Another interpretation from the API implementation perspective is to limit the number of level returns in one API call. But that is not Dggrs definition related.

There is a distinction between the DGGRS definition (as per the examples in Annex B linked with [ogc-rel:dggrs-definition]) and the DGGRS description (as in the resources at .../dggrs/{dggrsId}. Those are completely different objects. The DGGRS description, is specific to the particular implementation / deployment, and if for a particular collection, specific to that collection. In a DGGRS definition, tessellation could normally go on infinitely up to the Planck scale.

So these properties (of the DGGRS description) serve two completely different purposes:

maxRelativeDepth is the greatest relative depths that the server accepts to return for zone data query, independently of the level of the zone for which data is being requested, and independently of the native resolution of the data. For example, if requesting data from a level 0 zone, for data that goes down to level 20, if maxRelativeDepth is 10, the server will reject requests for .../dggs/{dggrsId}/zones/{zoneId-at-level-0}/data?zone-depth=11, even though 0 + 11 < 20.

maxRefinementLevel on the other hand would normally be specific to a particular dataset or collection, and indicates the native resolution of the data. It does not necessarily mean that the server will reject requests beyond this level (it could upsample the data), but it is allowed to do so -- see Permission 1. So in the example above, maxRefinementLevel would be 20, and for a GET request on .../dggs/{dggrsId}/zones/{zoneId-at-level-20}/data?zone-depth=10, the server might either upsample the data (represented internally by a single zone value) by 10 levels, or return an error. A GET request on the associated parent zone of level 10 for a relative depth of 10: .../dggs/{dggrsId}/zones/{zoneId-at-level-10}/data?zone-depth=10 would return the full resolution of the data, so if a client uses a relative zone-depth of 10 for its data requests, then there is no need to ever retrieve from (parent) zoneIds > 10 in this case.

@tik65536
Copy link
Author

tik65536 commented Dec 8, 2024

@jerstlouis Thx. If I understand correctly, it restricts the maximum zone level we can include in the response from the current zone level.
It is possible to include the above example on maxRelativeDepth in section 9.2 Recommendation 5 for future reference? I think it's better to illustrate it with an example.

@jerstlouis
Copy link
Member

jerstlouis commented Dec 8, 2024

it restricts the maximum zone level we can include in the response from the current zone level

For Zone Data Retrieval, the level of the responses should always be well-defined.

If the client did not specify a zone-depth parameter, the zone data response needs to be the value specified in the defaultDepth property of the DGGRS description (.../dggs/{dggrsId}) as specified in Requirement 4F.

If the server does not conform to the "Custom Depths" requirement class, that is the only valid option.

If the client did specify a zone-depth beyond the maxRelativeDepth, it should be a 4xx error as specified in Recommendation 5.

If the client specified a valid zone-depth (and the server supports Custom Depths), the response should respect that zone depth, unless the negotiated format does not support that particular zone-depth. For example, some formats may only support a particular depth, or only support a single depth rather than a range of depths.

It is possible to include the above example on maxRelativeDepth in section 9.2 Recommendation 5 for future reference? I think it's better to illustrate it with an example.

We could add some examples just below the recommendation if that would be useful.

@tik65536
Copy link
Author

tik65536 commented Dec 9, 2024

it restricts the maximum zone level we can include in the response from the current zone level

For Zone Data Retrieval, the level of the responses should always be well-defined.

If the client did not specify a zone-depth parameter, the zone data response needs to be the value specified in the defaultDepth property of the DGGRS description (.../dggs/{dggrsId}) as specified in Requirement 4F.

Sorry, it causes confusion for me right now.
The data retrieval endpoint comes with zoneID, and the DGGRS should determine the zone level from the zone ID requested if zone-depth is not defined, right?

Let's, for example dggrid with z7string. It determines the zone level by the number of strings (omit the first 2 from left)

Example:

/dggrs/IGEO7/0023262345/data, so the zone level will be 8 (8 chars: 23262345)

If the server does not conform to the "Custom Depths" requirement class, that is the only valid option.

If the client did specify a zone-depth beyond the maxRelativeDepth, it should be a 4xx error as specified in Recommendation 5.

If the client specified a valid zone-depth (and the server supports Custom Depths), the response should respect that zone depth, unless the negotiated format does not support that particular zone-depth. For example, some formats may only support a particular depth, or only support a single depth rather than a range of depths.

Carry from the above assumption (zone level is determined from zone ID); the zone-depth parameter will be the relative zone level based on the base zone level.

For example :

Example: /dggrs/IGEO7/0023262345/data?zone-depth=2

It will return data with zone level 10 (8 + 2), and if the maxRelativeLevel is set with 11, then it should be a valid request.
But with maxRelativeLevel=9, then it should be rejected.

That's the logic from your first reply.

It is possible to include the above example on maxRelativeDepth in section 9.2 Recommendation 5 for future reference? I think it's better to illustrate it with an example.

We could add some examples just below the recommendation if that would be useful.

@jerstlouis
Copy link
Member

jerstlouis commented Dec 9, 2024

@tik65536

/dggrs/IGEO7/0023262345/data, so the zone level will be 8 (8 chars: 23262345)

8 from the {zoneId} is the level of the parent clipping zone for which data is being requested.

The data has to be returned at 8 + defaultDepth -- whatever defaultDepth is set to in .../dggs/{dggrsId} --
defaultDepth should never be 0 because returning a single value per API request is extremely inefficient and silly.

Depending on the refinement ratio of your DGGRS, different defaultDepth values might be more appropriate.

We try to return ~64k values by default, similar to typical 256x256 tiles. That's a defaultDepth of 10 for ISEA3H, 5 for ISEA9R and 8 for GNOSISGlobalGrid. For ISEA7H / H3, you could pick 5 (7^5 = 16,807) or 6 (7^6 = 117,649).

The level of the parent clipping zone (to which the relative depth is added to obtain the absolute depth of the sub-zones for which individual values are being returned) is always inferred from the {zoneId}, whether the zone-depth query parameter is used or not.

@jerstlouis
Copy link
Member

jerstlouis commented Dec 9, 2024

@tik65536

It will return data with zone level 10 (8 + 2), and if the maxRelativeLevel is set with 11, then it should be a valid request.
But with maxRelativeLevel=9, then it should be rejected.

That is not correct. The relative depth here is the 2 specified in zone-depth=2, so whether maxRelativeDepth (not maxRelativeLevel) is 9 or 11 does not matter -- that's still greater than 2.

If you meant maxRefinementLevel, then the server would be allowed to either reject it or upsample if that's set to 9 per Permission 1.

@tik65536
Copy link
Author

tik65536 commented Dec 10, 2024

Hey ~ Thx.

I misunderstood the spec at the very beginning.
So, the client will always need to consider what level is needed to query the data by their parent ID (required level - default depth or zone-depth).

So, in terms of the flow on how to retrieve data:

  1. First, you can get the list of zone IDs by /dggs/IGEO7/zones with bbox and required zone level (ex. 8 - (default depth/zone-depth ))
  2. Then you get the data by /dggs/IGEO7/zones/{zoneID}/data ( it will return parent zone level + (default depth/zone-depth)
  3. Then, perform aggregation on the client side if needed.
  • No matter which logic to be use ( relativeDepth or return with the same level), the data will be the same. just how the data is partitioned (aggregated) is different (ie. partition by resolution 8 (same level ), or 9 ( if defaultDepth is 1)

However, the point you mentioned above on returning one value per API is not the main reason. For example, multiple data can still be associated with zone ID at level 8; it depends on whether the data provider will aggregate the data before return.
By the way, do we assume (or stated in the spec) that the data must be aggregated before return?

In general, it will benefit development if the API are more intuitive ( like what you sent, what you will get).

For the above logic, I suggest aligning the other's API endpoint with {zoneID} supported to have the same logic. Otherwise, it confuses people a lot.

Thx , Tik

@jerstlouis
Copy link
Member

jerstlouis commented Dec 11, 2024

I suggest aligning the other's API endpoint with {zoneID} supported to have the same logic. Otherwise, it confuses people a lot.

I'm not clear on where the confusion lies, or what you mean by the first sentence. I do not believe significant API changes are justified or necessary at this stage. First we should discuss to clarify how the API is intended to be used, then you could help us facilitate readers' understanding by pointing out where we need additional clarifications.

So, the client will always need to consider what level is needed to query the data by their parent ID (required level - default depth or zone-depth).

In general, it will benefit development if the API are more intuitive ( like what you sent, what you will get).

The Zone Data Retrieval is intended to retrieve multiple data values in a single request.
Retrieving individual values for single zones one at a time has no practical use due to the heavy cost of round-trip HTTP requests.

So necessarily, there is a parent level for the zone of interest -- the {zoneId} for the .../dggs/zones{zoneID}/data request, and a relative depth, which added to the parent level, yields the absolute level of the sub-zones for which values are being returned. This data packet containing sub-zones is still conceptually a data packet for the {zoneId}.

  1. First, you can get the list of zone IDs by /dggs/IGEO7/zones with bbox and required zone level (ex. 8 - (default depth/zone-depth ))

For a DGGS API client with a built-in understanding of a DGGRS supported by the server, for a pure data visualization scenario, there would be no need to first perform a Zone Query. The client would instead use its own library to figure out which zones at which level it wishes to visualize. If the client is 3D, the region of interest would likely not be a bounding box or at the same refinement level either. When doing this, the client would again pick zones of a coarser level compared to the number of pixels it wishes to represent distinctly on the screen. If we take the example of ISEA3H hexagons with a depth of 10 containing about 64k sub-zones, it could compute hexagons at refinement levels where each hexagon that is displayed on the screen occupies roughly about 64k pixels, before retrieving data at level 10 for each of these larger zones. Requesting zones individually would mean 64,000 times more HTTP requests (!).

Conceptually, the Zone Data retrieval is very similar to OGC API - Tiles and WMTS, with the exception that DGGRS are much more flexible than 2D Tile Matrix Sets e.g., zones do not need to be rectangular.

If we take the WorldCRS84Quad 2DTMS for example, which covers the whole world in OGC:CRS84, and a corresponding hypothetical DGGRS (the GNOSISGlobalGrid is actually very similar to this, except for starting at level 1 of WorldCRS84Quad and handling the polar regions better). WorldCRS84Quad, like many 2DTMS, defines tiles as being 256x256. This would correspond to a defaultDepth or zone-depth of 8. So just like when retrieving a tile at/tiles/{tmsId}/{level}/{row}/{column} returns 65536 values, retrieving zone data from a zone at /dggs/{dggrsId}/zones/{zoneId}/data?zone-depth=8 would return zone data with 65536 values.

The sub-zones of DGGS Zone Data corresponds to the individual pixels or cells inside the tile, whereas the {zoneId} parent zone corresponds to the {level}/{row}/{column} tile.

There is a very similar analogy between the internal grid of Mapbox Vector Tiles and the sub-zones used to quantize coordinates of vector geometry with DGGS-FG-JSON.

There is also no concept of default or relative zone depth with .../dggs/{dggrsId}/zones Zone Query. There is a zone-level parameter which specifies which level of zone should be returned (the precision of the query). The compact-zones parameter should also be set to false if you do not want completely filled parents of a coarser refinement level to replace the children.

  1. Then, perform aggregation on the client side if needed.

The client could potentially perform aggregation as a use case, but I don't understand why you bring up aggregation here?
DGGS-quantized raster data stores are naturally a hierarchy of aggregated data.
So a client wishing for a coarser resolution could either request a lower zone-depth (if the server supports Custom Zone Depths) -- but not too low so as to need too many requests, or it could use a coarser parent {zoneId} for the request so that the individual sub-zone values are at the desired refinement level.

For example, multiple data can still be associated with zone ID at level 8; it depends on whether the data provider will aggregate the data before return. By the way, do we assume (or stated in the spec) that the data must be aggregated before return?

The assumption for Zone Data Retrieval is that the data is quantized to the sub-zone level -- sub-zone level = parent level from {zoneId} + (defaultDepth or zone-depth), and ideally (recommendation) clipped to the {zoneId} parent zone's geometry.

For the DGGS-JSON (raster) encoding (and I would expect the same for Zarr), this means one value (or set of properties/variables/fields) per sub-zone. The value for coarser refinement level would normally be some aggregation of the corresponding values for finer refinement levels.

For the DGGS-FG-JSON (vector) encoding, this means that geometry points are precise to the sub-zone. The properties associated with vector features would usually be the same at all refinement levels (with potential exceptions for special generalization strategies), but the geometry naturally gets generalized by the coarser precision of the sub-zones.

Representations of Zone Data in other formats are for facilitating interoperability with non-DGGS clients, and the API implementations do their best to return an equivalent representation of the data constrained by the limitations of the particular format.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants