Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify save_result #289

Merged
merged 5 commits into from
Nov 4, 2021
Merged

Clarify save_result #289

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1f962df
Clarify save_result #288
m-mohr Oct 25, 2021
a2cc14a
Update CHANGELOG.md
m-mohr Oct 27, 2021
46e4f5c
Merge branch 'draft' into issue-288
m-mohr Oct 27, 2021
a5fceaa
Improvements from code review
m-mohr Oct 28, 2021
15b90cc
Improvement from code review
m-mohr Nov 4, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed

- `aggregate_temporal_period`: Clarified which dimension labels are present in the returned data cube. [#274](https://github.com/Open-EO/openeo-processes/issues/274)
- `ard_surface_reflectance`: has been categorized as "optical" instead of "sar".
- `ard_surface_reflectance`: The process has been categorized as "optical" instead of "sar".
- `save_result`: Clarify how the process works in the different contexts it is used in (e.g. synchronous processing, secondary web service). [#288](https://github.com/Open-EO/openeo-processes/issues/288)
- `quantiles`: Clarified behavior. [#278](https://github.com/Open-EO/openeo-processes/issues/278)

## [1.1.0] - 2021-06-29
Expand Down
10 changes: 5 additions & 5 deletions save_result.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
{
"id": "save_result",
"summary": "Save processed data to storage",
"description": "Saves processed data to the server-side user workspace of the authenticated user. This process aims to be compatible with GDAL/OGR formats and options. STAC-compatible metadata should be stored with the processed data.\n\nCalling this process may be rejected by back-ends in the context of secondary web services.",
"summary": "Save processed data",
"description": "Makes the processed data available in the given file format to the corresponding medium that is relevant for the context this processes is applied in:\n\n* For **batch jobs** the data is stored in the server-side user workspace of the authenticated user. STAC-compatible metadata are usually made available with the processed data.\n* For **synchronous processing** the data is directly sent to the client.\n* For secondary web services the behavior depends on the service type. Usually, the data is directly provided to the secondary web service, but calling this process may also be rejected by some service types.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For batch jobs the data is stored in the server-side user workspace of the authenticated user.

I might be missing something, but the concept "workspace" is only/mainly used for the file storage part of the API.
Batch job results are not necessarily stored in a way that's compatible with that file storage API (e.g. in VITO backend, the batch result are not stored in a user specific "home" folder or something like that)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For synchronous processing the data is directly sent to the client

minor note: the "directly" might be misleading because of additional zipping of the result.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For secondary web services the behavior depends on the service type. Usually, the data is directly provided to the secondary web service, but calling this process may also be rejected by some service types

I'm left confused about this

  • "directly provided": does that mean "skipping save_result"?
  • sentence starts with "usually .. directly provided ...", but what is the "otherwise" part? non-directly providing like a batch jobs?
  • what if service type requires save_result, but it is missing?

I guess it's not worth trying to explain all the secondary service aspects here? Maybe the description should be very high short and just refer to some general secondary services docs (e.g at https://openeo.org/documentation/1.0/developers/api/reference.html#tag/Secondary-Services)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • batch jobs / user workspace: Yes, it's not necessarily the same. We are also removing that language in load_result: Load by URL and filter by extents and bands #292, so this can be aligned. Simply removed it.

  • synchronous: Changed to

    For synchronous processing the data is sent to the client as a direct response to the request

  • sec. web services: Hmm, is this better?

    Secondary web services are provided with the processed data so that it can make use of it (e.g., visualize it). Web service may require the data in a certain format. Please refer to the documentation of the individual service types for details.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes looks good.

re: secondary services: I guess we don't have enough experience with that at the moment to have fully polished docs, so I'm fine with last proposal

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed.

"categories": [
"cubes",
"export"
],
"parameters": [
{
"name": "data",
"description": "The data to save.",
"description": "The data to make available in the given file format.",
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
"schema": [
{
"type": "object",
Expand All @@ -23,7 +23,7 @@
},
{
"name": "format",
"description": "The file format to save to. It must be one of the values that the server reports as supported output file formats, which usually correspond to the short GDAL/OGR codes. If the format is not suitable for storing the underlying data structure, a `FormatUnsuitable` exception will be thrown. This parameter is *case insensitive*.",
"description": "The file format to use. It must be one of the values that the server reports as supported output file formats, which usually correspond to the short GDAL/OGR codes. If the format is not suitable for storing the underlying data structure, a `FormatUnsuitable` exception will be thrown. This parameter is *case insensitive*.",
"schema": {
"type": "string",
"subtype": "output-format"
Expand All @@ -41,7 +41,7 @@
}
],
"returns": {
"description": "`false` if saving failed, `true` otherwise.",
"description": "Reurns `false` if the process failed to make the data available, `true` otherwise.",
m-mohr marked this conversation as resolved.
Show resolved Hide resolved
"schema": {
"type": "boolean"
}
Expand Down