Workshoping the Drafting Process of the OGC API DGGS Spec

[geofizzydrink]

Initial Topics for Elaboration

• What is in Core?
• What do we want to split out to additional parts?
• What is the granularity of the Conformance Classes?
• Need to assign all the the API functions/endpoints prototypes thus far to a specific conformance class
• Need to prioritise which endpoints/elements are included in version 1.0 and which (if any) will be held off to later revisions
• Scoping what CITE Tests are need

[geofizzydrink]

• Common
  • api
  • conformance
  • collections
  • collections/{collectionID}
• DGGS Core
  Question: how do we define transportable endpoints
  - Tiles API is a good example
  • dggs
    • list of dggsRSIDs available/provided by this server
  • dggs/{dggsRSID}
    • custom definition (as per ISO 19170) or a link to the OGC Registry Record for definition
    • "Recommendation" to include a DGGS definition rather than a "Requirement"
    • e.g. http://docs.ogc.org/DRAFTS/20-057.html#_response_4

< keep these for the "What is Here?" and "Where is it?" classes >

• dggs/{dggsRSID}/zones

  • how do we impose default values if a user doesn't provide query filter parameters?
  • Do we really need this endpoint at the highest/generic level that is not associated with data

• dggs/{dggsRSID}/zones/{ZonalID} (need to add the s to zone(s))

  • Do we really need this endpoint at the highest/generic level that is not associated with data

• What is Here? class

  • Data Retrieval conformance class - explore type of function but probably better to be funcitonal
  • can be modelled on Tiles API
  • /collections/{collectionId}/dggs/
    • list of dggsRSIDs mapped to this collection

  • /collections/{collectionId}/dggs/{dggsRSID}

  • /collections/{collectionId}/dggs/{dggsRSID}/zones/{zonalID}

• Where is it? class

  • Search conformance class
  • /collections/{collectionId}/dggs/
  • /collections/{collectionId}/dggs/{dggsRSID}
  • /collections/{collectionId}/dggs/{dggsRSID}/zones

• CQL conformance class - filtering

  • https://docs.opengeospatial.org/DRAFTS/21-065.html#spatial-operators
  • /collections/{collectionId}/dggs/{dggsRSID}/zones?filter=
  • include cql recipes for the topological (and an other) functions relevant to DGGS

[geofizzydrink]

• common class Plus

4 DGGS classes

• Core (question conformance class or section of spec/shared requirements)
• Data Retrieval
• Search
• Filtering
• Aggregation? (should this be separate to Filtering???)
• Analytics? (should this be separate to Filtering???)

https://docs.opengeospatial.org/DRAFTS/19-079r1.html

[jerstlouis]

@geofizzydrink @allixender @perrypeterson
As a follow up to today's meeting, see proposed new API definition and preview in SwaggerUI.

I plan to implement a simple working prototype of it as a next step.

In terms of the later steps, I think they are not necessary to complete Part 1: Core (Core, Data Retrieval: What is here?, Zone Query: Where is it?), but:

• The Filtering capability for Part 2 should be able to use CQL2 as-is, but might benefit from DGGS-specific extensions (e.g. to refer to list of zones). It would make sense to synchronize with Features - Part 3 and Coverages (See Retrieve values within a certain range ogcapi-coverages#103 and Coverage Processing Language ogcapi-coverages#146).
• For the Search capability (also in Part 2?), it would make sense to define a generic expression language derived from CQL2, and synchronize with the Search proposal for Features.
• The aggregation capability could be built directly into that Part 2 Search query language, but it would also make sense to synchronize with EDR and Coverages and consider the previous work from DAPA.
• A Part 3 could bring back the zones topological functions from the first iteration (although this might be more for testing / demonstration purposes than for practical client/server interactions).
• Advanced analytics could be supported simply by combining with Processes - Part 3: Workflows and Chaining Collection/DataSet Output (see also Workflow examples and use cases ... ogcapi-processes#279 (comment)).
• We could define compact zone data packets representations (for either single or multiple zones) for both coverage and feature data independently of the API specification.

[chris-little]

The EDR API SWG memeber have been experimenting successfully with integrating with API-Processes.
We are also looking to support categorical dimensions (domain, coordinate) for both retrieving and aggregation.
But on the other hand we do not want to re-invent python, R, Jupiter, etc
Some aggregation could be straightforward (e.g. time averageing or accumulation, vertical accumulation (e.g. total water column salinity, etc) but others not som clear (e.g. x without y not usually sensible or useful)

[BenJin77]

Since it is an API standard compatible with various DGGSs, should we consider adding functions for interoperability of different types of DGGSs?

[jerstlouis]

@BenJin77 One API implementation can support multiple DGGS at /dggs/{dggsId}.

However I think that the core of the API should focus on zone queries and data retrieval requests for one specific DGGS.

If the system supports multiple DGGS, it should still be possible to reference collections of data that may internally be stored using different DGGS, returning results according to one specific DGGS. Or to use non-DGGS APIs like Features, Coverages, EDR to return the results in a non-DGGS specific way. And collections= could also potentially support referencing collections available from remote APIs as well (on a different server).

Do you feel this would be missing any other interoperability functionality that you are considering?

[rggibb]

@BenJin77, @jerstlouis I think the issues of having a single query/analysis API that can address different DGGS, or indeed as well as other data types, is a serapate issue from the issue of data type conversion - which includes DGGS data conversion. The important thing as Jerome has stated is that the dggsId is explicit so that the zoneids are well defined and not ambiguous.

We've have talked previously of the need for a DGGS best practice, and I think conversion between different types of DGGS (ie differing dggsId) is a key topic for best practice. There are common statements we can make about data conversion, that cover both conversion between differing DGGS and from DGGS to/from non-DGGS. For instance in discussing conversion from satellite imagery to DGGS, the best practice seems to be to choose a destination resolution that is one step finer than the native non-DGGS resolution, and then potentially agregate the data back to a resolution one coarser in the hierarchy using an agregation statistic appropriate for the source' satellite imagery. This principle can be be applied to any source data type, not just satellite imagery, so the question becomes how much of this conversion do we want to explicitly encapsulate in a standard as distinct from a best practice? The way the standard is progressing towards datatype agnostic analytics, my gut feeling is that all the required filtering and agregation tools will already be present in the API for use in other circumstances eg Derived fields and aggregation support #164.

So with that in mind the only missing bits are the bits about choosing a destination resolution one step finer that the source resolution, the question about whether that resolution is saved for future use, or just temporary, and then the agregation to the next coarser resolution in the DGGS hierarchy. And of couase this is potentlially an instance of a derived field.

[jerstlouis]

Working out some more of the details for the derived fields and aggregation support I think:

• When properties (to select only some of the existing fields, or compute new fields, possibly using aggregration) is used with the Data Retrieval conformance class (.../dggs/{dggsId}/zones/{zoneId}/data or .../dggs/{dggsId}/data) -- it specifies what fields / properties will be included in the response (e.g., the GeoJSON, GeoTIFF, netCDF, Zarr file...). When aggregating over dimensions of the DGGS itself, the data retrieval response would return much smaller responses (e.g., a single record with values per zone).
• When properties is used with a Zone Query request (.../dggs/{dggsId}/zones), it could mean to include an aggregation in the response, along with the list of zones (assuming the zone list response format supports this). It could default to a Sum() over all dimensions, if no specific aggregation function is used in a particular field expression.
• An implementation could also decide to automatically return informative default aggregation of a collection's fields over the zone in the optional zone info (.../dggs/{dggsId}/zones/{zoneId}) by default, if they are quick to compute.

[jerstlouis]

The OGC API - DGGS candidate Standard is now a completed draft and in RFC period.

The latest draft follows the outline described by @geofizzydrink in #47 (comment) quite closely.

The topics of aggregation and analytics, beyond what is possible with CQL2 filtering for zone queries / zone data could be the topic of future extension parts. For example, the "derived field" capability would be useful. This capability is already in discussion in Coverages as well as in Features (opengeospatial/ogcapi-features#927), so we can wait to remain in alignment with these.

Recommendation 2 O of Zone Info covers the inclusion of statistic for a particular zone (/dggs/{dggrsId}/zones/{zoneId}), and Recommendation 6 of Zone query makes the recommendation of including the total area/volume of the query. Extensions could specify requirements for such functionality as well as more advanced analytics/aggregation capabilities. We should open new issues specific to those future parts.

Regarding @rggibb 's comment

So with that in mind the only missing bits are the bits about choosing a destination resolution one step finer that the source resolution

I think that the Zone Data Retrieval with its zone-depth query parameter and the fact that you can request data from any refinement level that the API allows to return (which could already potentially be beyond the source resolution) should already support that use case.

Therefore, closing this issue.