Release openEO processes v1.2.0

CHANGELOG.md

@@ -6,17 +6,59 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased / Draft
 
+## [1.2.0] - 2021-12-13
+
+### Added
+
+- New processes in proposal state
+    - `fit_curve`
+    - `predict_curve`
+- `ard_normalized_radar_backscatter` and `sar_backscatter`: Added `options` parameter
+- `array_find`: Added parameter `reverse`. [#269](https://github.com/Open-EO/openeo-processes/issues/269)
+- `load_result`:
+    - Added ability to load by (signed) URL (supported since openEO API v1.1.0).
+    - Added parameters `spatial_extent`, `temporal_extent` and `bands`. [#220](https://github.com/Open-EO/openeo-processes/issues/220)
+- `run_udf`: Exception `InvalidRuntime` added. [#273](https://github.com/Open-EO/openeo-processes/issues/273)
+- A new category "math > statistics" has been added [#277](https://github.com/Open-EO/openeo-processes/issues/277)
+
+### Changed
+
+- `array_labels`: Allow normal arrays to be passed for which the process returns the indices. [#243](https://github.com/Open-EO/openeo-processes/issues/243)
+- `debug`:
+    - Renamed to `inspect`.
+    - The log level `error` does not need to stop execution.
+    - Added proposals for logging several data types to the implementation guide.
+
+### Removed
+
+- Removed the explicit schema for `raster-cube` in the `data` parameters and return values of `run_udf` and `run_udf_externally`. It's still possible to pass raster-cubes via the "any" data type, but it's discouraged due to scalability issues. [#285](https://github.com/Open-EO/openeo-processes/issues/285)
+
+### Fixed
+
+- `aggregate_temporal_period`: Clarified which dimension labels are present in the returned data cube. [#274](https://github.com/Open-EO/openeo-processes/issues/274)
+- `ard_surface_reflectance`: The process has been categorized as "optical" instead of "sar".
+- `array_modify`: Clarified behavior.
+- `save_result`: Clarify how the process works in the different contexts it is used in (e.g. synchronous processing, secondary web service). [#288](https://github.com/Open-EO/openeo-processes/issues/288)
+- `quantiles`:
+  - The default algorithm for sample quantiles has been clarified (type 7). [#296](https://github.com/Open-EO/openeo-processes/issues/296)
+  - Improved documentation in general. [#278](https://github.com/Open-EO/openeo-processes/issues/278)
+
 ## [1.1.0] - 2021-06-29
 
 ### Added
+
 - New processes in proposal state
+    - `ard_normalized_radar_backscatter`
+    - `ard_surface_relectance`
     - `array_append`
     - `array_concat`
     - `array_create`
     - `array_create_labeled`
     - `array_find_label`
     - `array_interpolate_linear` [#173](https://github.com/Open-EO/openeo-processes/issues/173)
     - `array_modify`
+    - `atmospheric_correction`
+    - `cloud_detection`
     - `date_shift`
     - `is_infinite`
     - `nan`
@@ -30,7 +72,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
     - Moved the experimental process `run_udf_externally` to the proposals.
     - Moved the rarely used and implemented processes `cummax`, `cummin`, `cumproduct`, `cumsum`, `debug`, `filter_labels`, `load_result`, `load_uploaded_files`, `resample_cube_temporal` to the proposals.
 - Exception messages have been aligned always use ` instead of '. Tooling could render it with CommonMark.
-- `load_collection`  and `mask_polygon`: Also support multi polygons instead of just polygons. [#237](https://github.com/Open-EO/openeo-processes/issues/237)
+- `load_collection` and `mask_polygon`: Also support multi polygons instead of just polygons. [#237](https://github.com/Open-EO/openeo-processes/issues/237)
 - `run_udf` and `run_udf_externally`: Specify specific (extensible) protocols for UDF URIs.
 - `resample_cube_spatial` and `resample_spatial`: Aligned with GDAL and added `rms` and `sum` options to methods. Also added better descriptions.
 - `resample_cube_temporal`: Process has been simplified and only offers the nearest neighbor method now. The `process` parameter has been removed, the `dimension` parameter was made less restrictive, the parameter `valid_within` was added. [#194](https://github.com/Open-EO/openeo-processes/issues/194)
@@ -221,7 +263,8 @@ First version which is separated from the openEO API. Complete rework of all pro
 Older versions of the processes were released as part of the openEO API, see the corresponding changelog for more information.
 
 
-[Unreleased]: <https://github.com/Open-EO/openeo-processes/compare/1.1.0...HEAD>
+[Unreleased]: <https://github.com/Open-EO/openeo-processes/compare/1.2.0...HEAD>
+[1.2.0]: <https://github.com/Open-EO/openeo-processes/compare/1.1.0...1.2.0>
 [1.1.0]: <https://github.com/Open-EO/openeo-processes/compare/1.0.0...1.1.0>
 [1.0.0]: <https://github.com/Open-EO/openeo-processes/compare/1.0.0-rc.1...1.0.0>
 [1.0.0-rc.1]: <https://github.com/Open-EO/openeo-processes/compare/0.4.2...1.0.0-rc.1>

README.md

@@ -8,12 +8,13 @@ openEO develops interoperable processes for big Earth observation cloud processi
 
 The [master branch](https://github.com/Open-EO/openeo-processes/tree/master) is the 'stable' version of the openEO processes specification. An exception is the [`proposals`](proposals/) folder, which provides experimental new processes currently under discussion. They may still change, but everyone is encouraged to implement them and give feedback.
 
-The latest release is version **1.1.0**. The [draft branch](https://github.com/Open-EO/openeo-processes/tree/draft) is where active development takes place. PRs should be made against the draft branch.
+The latest release is version **1.2.0**. The [draft branch](https://github.com/Open-EO/openeo-processes/tree/draft) is where active development takes place. PRs should be made against the draft branch.
 
 | Version / Branch                                             | Status                    | openEO API versions |
 | ------------------------------------------------------------ | ------------------------- | ------------------- |
 | [unreleased / draft](https://processes.openeo.org/draft)     | in development            | 1.x.x               |
-| [**1.1.0** / master](https://processes.openeo.org/1.1.0/)    | **latest stable version** | 1.x.x               |
+| [**1.2.0** / master](https://processes.openeo.org/1.2.0/)    | **latest stable version** | 1.x.x               |
+| [1.1.0](https://processes.openeo.org/1.1.0/)                 | legacy version            | 1.x.x               |
 | [1.0.0](https://processes.openeo.org/1.0.0/)                 | legacy version            | 1.x.x               |
 | [1.0.0 RC1](https://processes.openeo.org/1.0.0-rc.1/)        | legacy version            | 1.x.x               |
 | [0.4.2](https://processes.openeo.org/0.4.2/)                 | legacy version            | 0.4.x               |
@@ -33,11 +34,7 @@ This repository contains a set of files formally describing the openEO Processes
 * [implementation.md](meta/implementation.md) in the `meta` folder provide some additional implementation details for back-ends. For back-end implementors, it's highly recommended to read them.
 * [subtype-schemas.json](meta/subtype-schemas.json) in the `meta` folder defines common data types (`subtype`s) for JSON Schema used in openEO processes.
 * The [`examples`](examples/) folder contains some useful examples that the processes link to. All of these are non-binding additions.
-* The [`tests`](tests/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser.
-
-	If you switch to the `tests` folder in CLI and after installing NodeJS and run `npm install`, you can run a couple of commands:
-	* `npm test`: Check the processes for validity and lint them. Processes need to pass tests to be added to this repository.
-	* `npm run render`: Opens a browser with all processes rendered through the docgen.
+* The [`tests`](tests/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser. Check the [tests documentation](tests/README.md) for details.
 
 ## Process
 

aggregate_temporal_period.json

@@ -10,7 +10,7 @@
     "parameters": [
         {
             "name": "data",
-            "description": "A data cube.",
+            "description": "The source data cube.",
             "schema": {
                 "type": "object",
                 "subtype": "raster-cube"
@@ -73,7 +73,7 @@
         },
         {
             "name": "dimension",
-            "description": "The name of the temporal dimension for aggregation. All data along the dimension is passed through the specified reducer. If the dimension is not set or set to `null`, the data cube is expected to only have one temporal dimension. Fails with a `TooManyDimensions` exception if it has more dimensions. Fails with a `DimensionNotAvailable` exception if the specified dimension does not exist.",
+            "description": "The name of the temporal dimension for aggregation. All data along the dimension is passed through the specified reducer. If the dimension is not set or set to `null`, the source data cube is expected to only have one temporal dimension. Fails with a `TooManyDimensions` exception if it has more dimensions. Fails with a `DimensionNotAvailable` exception if the specified dimension does not exist.",
             "schema": {
                 "type": [
                     "string",
@@ -94,7 +94,7 @@
         }
     ],
     "returns": {
-        "description": "A new data cube with the same dimensions. The dimension properties (name, type, labels, reference system and resolution) remain unchanged, except for the resolution and dimension labels of the given temporal dimension. The specified temporal dimension has the following dimension labels (`YYYY` = four-digit year, `MM` = two-digit month, `DD` two-digit day of month):\n\n* `hour`: `YYYY-MM-DD-00` - `YYYY-MM-DD-23`\n* `day`: `YYYY-001` - `YYYY-365`\n* `week`: `YYYY-01` - `YYYY-52`\n* `dekad`: `YYYY-00` - `YYYY-36`\n* `month`: `YYYY-01` - `YYYY-12`\n* `season`: `YYYY-djf` (December - February), `YYYY-mam` (March - May), `YYYY-jja` (June - August), `YYYY-son` (September - November).\n* `tropical-season`: `YYYY-ndjfma` (November - April), `YYYY-mjjaso` (May - October).\n* `year`: `YYYY`\n* `decade`: `YYY0`\n* `decade-ad`: `YYY1`",
+        "description": "A new data cube with the same dimensions. The dimension properties (name, type, labels, reference system and resolution) remain unchanged, except for the resolution and dimension labels of the given temporal dimension. The specified temporal dimension has the following dimension labels (`YYYY` = four-digit year, `MM` = two-digit month, `DD` two-digit day of month):\n\n* `hour`: `YYYY-MM-DD-00` - `YYYY-MM-DD-23`\n* `day`: `YYYY-001` - `YYYY-365`\n* `week`: `YYYY-01` - `YYYY-52`\n* `dekad`: `YYYY-00` - `YYYY-36`\n* `month`: `YYYY-01` - `YYYY-12`\n* `season`: `YYYY-djf` (December - February), `YYYY-mam` (March - May), `YYYY-jja` (June - August), `YYYY-son` (September - November).\n* `tropical-season`: `YYYY-ndjfma` (November - April), `YYYY-mjjaso` (May - October).\n* `year`: `YYYY`\n* `decade`: `YYY0`\n* `decade-ad`: `YYY1`\n\nThe dimension labels in the new data cube are complete for the whole extent of the source data cube. For example, if `period` is set to `day` and the source data cube has two dimension labels at the beginning of the year (`2020-01-01`) and the end of a year (`2020-12-31`), the process returns a data cube with 365 dimension labels (`2020-001`, `2020-002`, ..., `2020-365`). In contrast, if `period` is set to `day` and the source data cube has just one dimension label `2020-01-05`, the process returns a data cube with just a single dimension label (`2020-005`).",
         "schema": {
             "type": "object",
             "subtype": "raster-cube"

array_apply.json

@@ -96,13 +96,13 @@
         {
             "rel": "example",
             "type": "application/json",
-            "href": "https://processes.openeo.org/1.1.0/examples/array_find_nodata.json",
+            "href": "https://processes.openeo.org/1.2.0/examples/array_find_nodata.json",
             "title": "Find no-data values in arrays"
         },
         {
             "rel": "example",
             "type": "application/json",
-            "href": "https://processes.openeo.org/1.1.0/examples/array_contains_nodata.json",
+            "href": "https://processes.openeo.org/1.2.0/examples/array_contains_nodata.json",
             "title": "Check for no-data values in arrays"
         }
     ]

array_contains.json

@@ -133,7 +133,7 @@
         {
             "rel": "example",
             "type": "application/json",
-            "href": "https://processes.openeo.org/1.1.0/examples/array_contains_nodata.json",
+            "href": "https://processes.openeo.org/1.2.0/examples/array_contains_nodata.json",
             "title": "Check for no-data values in arrays"
         }
     ],

array_find.json

@@ -1,7 +1,7 @@
 {
     "id": "array_find",
     "summary": "Get the index for a value in an array",
-    "description": "Checks whether the array specified for `data` contains the value specified in `value` and returns the zero-based index for the first match. If there's no match, `null` is returned.\n\n**Remarks:**\n\n* To get a boolean value returned use ``array_contains()``.\n* All definitions for the process ``eq()`` regarding the comparison of values apply here as well. A `null` return value from ``eq()`` is handled exactly as `false` (no match).\n* Data types MUST be checked strictly. For example, a string with the content *1* is not equal to the number *1*.\n* An integer *1* is equal to a floating-point number *1.0* as `integer` is a sub-type of `number`. Still, this process may return unexpectedly `false` when comparing floating-point numbers due to floating-point inaccuracy in machine-based computation.\n* Temporal strings are treated as normal strings and MUST NOT be interpreted.\n* If the specified value is an array, object or null, the process always returns `null`. See the examples for one to find `null` values.",
+    "description": "Returns the zero-based index of the first (or last) occurrence of the value specified by `value` in the array specified by `data` or `null` if there is no match. Use the parameter `reverse` to switch from the first to the last match.\n\n**Remarks:**\n\n* Use ``array_contains()`` to check if an array contains a value regardless of the position.\n* Use ``array_find_label()`` to find the index for a label.\n* All definitions for the process ``eq()`` regarding the comparison of values apply here as well. A `null` return value from ``eq()`` is handled exactly as `false` (no match).\n* Data types MUST be checked strictly. For example, a string with the content *1* is not equal to the number *1*.\n* An integer *1* is equal to a floating-point number *1.0* as `integer` is a sub-type of `number`. Still, this process may return unexpectedly `false` when comparing floating-point numbers due to floating-point inaccuracy in machine-based computation.\n* Temporal strings are treated as normal strings and MUST NOT be interpreted.\n* If the specified value is an array, object or null, the process always returns `null`. See the examples for one to find `null` values.",
     "categories": [
         "arrays",
         "reducer"
@@ -23,6 +23,15 @@
             "schema": {
                 "description": "Any data type is allowed."
             }
+        },
+        {
+            "name": "reverse",
+            "description": "By default, this process finds the index of the first match. To return the index of the last match instead, set this flag to `true`.",
+            "schema": {
+                "type": "boolean"
+            },
+            "default": false,
+            "optional": true
         }
     ],
     "returns": {
@@ -43,12 +52,28 @@
                 "data": [
                     1,
                     2,
+                    3,
+                    2,
                     3
                 ],
                 "value": 2
             },
             "returns": 1
         },
+        {
+            "arguments": {
+                "data": [
+                    1,
+                    2,
+                    3,
+                    2,
+                    3
+                ],
+                "value": 2,
+                "reverse": true
+            },
+            "returns": 3
+        },
         {
             "arguments": {
                 "data": [
@@ -139,7 +164,7 @@
         {
             "rel": "example",
             "type": "application/json",
-            "href": "https://processes.openeo.org/1.1.0/examples/array_find_nodata.json",
+            "href": "https://processes.openeo.org/1.2.0/examples/array_find_nodata.json",
             "title": "Find no-data values in arrays"
         }
     ]

array_labels.json

@@ -1,25 +1,21 @@
 {
     "id": "array_labels",
     "summary": "Get the labels for an array",
-    "description": "Gives all labels for a labeled array. The labels have the same order as in the array.",
+    "description": "Gives all labels for a labeled array or gives all indices for an array without labels. If the array is not labeled, an array with the zero-based indices is returned. The labels or indices have the same order as in the array.",
     "categories": [
         "arrays"
     ],
     "parameters": [
         {
             "name": "data",
-            "description": "An array with labels.",
+            "description": "An array.",
             "schema": {
-                "type": "array",
-                "subtype": "labeled-array",
-                "items": {
-                    "description": "Any data type."
-                }
+                "type": "array"
             }
         }
     ],
     "returns": {
-        "description": "The labels as an array.",
+        "description": "The labels or indices as array.",
         "schema": {
             "type": "array",
             "items": {

count.json

@@ -4,6 +4,7 @@
     "description": "Gives the number of elements in an array that matches the specified condition.\n\n**Remarks:**\n\n* Counts the number of valid elements by default (`condition` is set to `null`). A valid element is every element for which ``is_valid()`` returns `true`.\n* To count all elements in a list set the `condition` parameter to boolean `true`.",
     "categories": [
         "arrays",
+        "math > statistics",
         "reducer"
     ],
     "parameters": [

extrema.json

@@ -3,7 +3,7 @@
     "summary": "Minimum and maximum values",
     "description": "Two element array containing the minimum and the maximum values of `data`.\n\nThis process is basically an alias for calling both ``min()`` and ``max()``, but may be implemented more performant by back-ends as it only needs to iterate over the data once instead of twice.",
     "categories": [
-        "math"
+        "math > statistics"
     ],
     "parameters": [
         {

filter_temporal.json

@@ -1,6 +1,6 @@
 {
     "id": "filter_temporal",
-    "summary": "Temporal filter for a temporal intervals",
+    "summary": "Temporal filter based on temporal intervals",
     "description": "Limits the data cube to the specified interval of dates and/or times.\n\nMore precisely, the filter checks whether each of the temporal dimension labels is greater than or equal to the lower boundary (start date/time) and less than the value of the upper boundary (end date/time). This corresponds to a left-closed interval, which contains the lower boundary but not the upper boundary.",
     "categories": [
         "cubes",