Skip to content

Commit

Permalink
Add more subtype schema tests, improve test README, minor improvements
Browse files Browse the repository at this point in the history
  • Loading branch information
m-mohr committed Nov 29, 2021
1 parent 6a10a71 commit 3faf89d
Show file tree
Hide file tree
Showing 8 changed files with 135 additions and 45 deletions.
6 changes: 1 addition & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,11 +33,7 @@ This repository contains a set of files formally describing the openEO Processes
* [implementation.md](meta/implementation.md) in the `meta` folder provide some additional implementation details for back-ends. For back-end implementors, it's highly recommended to read them.
* [subtype-schemas.json](meta/subtype-schemas.json) in the `meta` folder defines common data types (`subtype`s) for JSON Schema used in openEO processes.
* The [`examples`](examples/) folder contains some useful examples that the processes link to. All of these are non-binding additions.
* The [`tests`](tests/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser.

If you switch to the `tests` folder in CLI and after installing NodeJS and run `npm install`, you can run a couple of commands:
* `npm test`: Check the processes for validity and lint them. Processes need to pass tests to be added to this repository.
* `npm run render`: Opens a browser with all processes rendered through the docgen.
* The [`tests`](tests/) folder can be used to test the process specification for validity and consistent "style". It also allows rendering the processes in a web browser. Check the [tests documentation](tests/README.md) for details.

## Process

Expand Down
14 changes: 8 additions & 6 deletions meta/subtype-schemas.json
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@
"type": "object",
"subtype": "chunk-size",
"title": "Chunk Size",
"description": "The chunk size per dimension given. This object maps the dimension names given as key to chunks given as either a physical measure or pixels. If not given or `null`, no chunking is applied.",
"required": [
"dimension",
"value"
Expand Down Expand Up @@ -108,7 +109,7 @@
"type": "string",
"subtype": "collection-id",
"title": "Collection ID",
"description": "A collection id from the list of supported collections.",
"description": "A collection identifier from the list of supported collections.",
"pattern": "^[\\w\\-\\.~/]+$"
},
"date": {
Expand All @@ -128,6 +129,7 @@
"duration": {
"type": "string",
"subtype": "duration",
"title": "Duration",
"description": "[ISO 8601 duration](https://en.wikipedia.org/wiki/ISO_8601#Durations), e.g. `P1D` for one day.",
"pattern": "^(-?)P(?=\\d|T\\d)(?:(\\d+)Y)?(?:(\\d+)M)?(?:(\\d+)([DW]))?(?:T(?:(\\d+)H)?(?:(\\d+)M)?(?:(\\d+(?:\\.\\d+)?)S)?)?$"
},
Expand Down Expand Up @@ -172,7 +174,7 @@
"type": "string",
"subtype": "input-format",
"title": "Input File Format",
"description": "An input format supported by the back-end."
"description": "A file format that the back-end supports to import data from."
},
"input-format-options": {
"type": "object",
Expand All @@ -191,7 +193,7 @@
"type": "array",
"subtype": "kernel",
"title": "Image Kernel",
"description": "Image kernel, a two-dimensional array of numbers.",
"description": "A two-dimensional array of numbers to be used as kernel for the image operation.",
"items": {
"type": "array",
"items": {
Expand Down Expand Up @@ -237,7 +239,7 @@
"type": "string",
"subtype": "output-format",
"title": "Output File Format",
"description": "An output format supported by the back-end."
"description": "A file format that the back-end supports to save and export data to."
},
"output-format-options": {
"type": "object",
Expand Down Expand Up @@ -390,13 +392,13 @@
"type": "string",
"subtype": "udf-runtime",
"title": "UDF runtime",
"description": "The name of a UDF runtime."
"description": "The identifier of a UDF runtime you want to run the given UDF source code with."
},
"udf-runtime-version": {
"type": "string",
"subtype": "udf-runtime-version",
"title": "UDF Runtime version",
"description": "The version of a UDF runtime."
"description": "The version of the UDF runtime you want to run the given UDF source code with."
},
"uri": {
"type": "string",
Expand Down
31 changes: 29 additions & 2 deletions tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,32 @@ To run the tests follow these steps:

1. Install [node and npm](https://nodejs.org) - should run with any recent version
2. Run `npm install` in this folder to install the dependencies
3. Run the tests with `npm test`.
4. To show the files nicely formatted in a web browser, run `npm run render`. It starts a server and opens the corresponding page in a web browser.
3. Run the tests with `npm test`. This will also lint the files and verify it follows best practices.
4. To show the files nicely formatted in a web browser, run `npm run render`. It starts a server and opens the corresponding page in a web browser.

## Development processes

All new processes must be added to the `proposals` folder. Each process must be declared to be `experimental`.
Processes must comply to best practices, which ensure a certain degree of consistency.
`npm test` will validate and lint the processes and also ensure the best practices are applied.

The linting checks that the files are named correctly, that the content is correctly formatted and indented (JSON and embedded CommonMark).
The best practices ensure that for examples the fields are not too short and also not too long for example.

A spell check is also checking the texts. It may report names and rarely used technical words as errors.
If you are sure that these are correct, you can add them to the `.words` file to exclude the word from being reported as an error.
The file must contain one word per line.

New processes should be added via GitHub Pull Requests.

## Subtype schemas

Sometimes it is useful to define a new "data type" on top of the JSON types (number, string, array, object, ...).
For example, a client could make a select box with all collections available by adding a subtype `collection-id` to the JSON type `string`.
If you think a new subype should be added, you need to add it to the `meta/subtype-schemas.json` file.
It must be a valid JSON Schema. The tests mentioned above will also verify to a certain degree that the subtypes are defined correctly.

## Examples

To get out of proposal state, at least two examples must be provided.
The examples are located in the `examples` folder and will also be validated to some extent in the tests.
17 changes: 8 additions & 9 deletions tests/processes.test.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
const glob = require('glob');
const fs = require('fs');
const path = require('path');
const { normalizeString, checkDescription, checkSpelling, checkJsonSchema, getAjv, prepareSchema } = require('./testHelpers');
const { normalizeString, checkDescription, checkSpelling, checkJsonSchema, getAjv, prepareSchema, isObject } = require('./testHelpers');

const anyOfRequired = [
"quantiles",
Expand Down Expand Up @@ -66,7 +66,7 @@ describe.each(processes)("%s", (file, p, fileContent, proposal) => {
// description
expect(typeof p.description).toBe('string');
// lint: Description should be longer than a summary
expect(p.description.length).toBeGreaterThan(55);
expect(p.description.length).toBeGreaterThan(60);
checkDescription(p.description, p);
});

Expand Down Expand Up @@ -98,7 +98,7 @@ describe.each(processes)("%s", (file, p, fileContent, proposal) => {
}

test("Return Value", () => {
expect(typeof p.returns).toBe('object');
expect(isObject(p.returns)).toBeTruthy();
expect(p.returns).not.toBeNull();

// return value description
Expand All @@ -108,14 +108,14 @@ describe.each(processes)("%s", (file, p, fileContent, proposal) => {
checkDescription(p.returns.description, p);

// return value schema
expect(typeof p.returns.schema).toBe('object');
expect(p.returns.schema).not.toBeNull();
expect(typeof p.returns.schema).toBe('object');
// lint: Description should not be empty
checkJsonSchema(jsv, p.returns.schema);
});

test("Exceptions", () => {
expect(typeof p.exceptions === 'undefined' || (typeof p.exceptions === 'object' && p.exceptions !== 'null')).toBeTruthy();
expect(typeof p.exceptions === 'undefined' || isObject(p.exceptions)).toBeTruthy();
});

var exceptions = o2a(p.exceptions);
Expand Down Expand Up @@ -153,7 +153,7 @@ describe.each(processes)("%s", (file, p, fileContent, proposal) => {
}
var paramKeys = Object.keys(parametersObj);

expect(typeof example).toBe('object');
expect(isObject(example)).toBeTruthy();
expect(example).not.toBeNull();

// example title
Expand Down Expand Up @@ -194,8 +194,7 @@ describe.each(processes)("%s", (file, p, fileContent, proposal) => {

if (Array.isArray(p.links)) {
test.each(p.links)("Links > %#", (link) => {
expect(typeof link).toBe('object');
expect(link).not.toBeNull();
expect(isObject(link)).toBeTruthy();

// link href
expect(typeof link.href).toBe('string');
Expand Down Expand Up @@ -250,8 +249,8 @@ function checkParam(param, p, checkCbParams = true) {
checkFlags(param);

// Parameter schema
expect(typeof param.schema).toBe('object');
expect(param.schema).not.toBeNull();
expect(typeof param.schema).toBe('object');
checkJsonSchema(jsv, param.schema);

if (!checkCbParams) {
Expand Down
22 changes: 0 additions & 22 deletions tests/subtype-schemas.test.js

This file was deleted.

29 changes: 29 additions & 0 deletions tests/subtypes-file.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
const fs = require('fs');
const $RefParser = require("@apidevtools/json-schema-ref-parser");
const { checkJsonSchema, getAjv, isObject, normalizeString } = require('./testHelpers');

test("File subtype-schemas.json", async () => {
let schema;
let fileContent;
try {
fileContent = fs.readFileSync('../meta/subtype-schemas.json');
schema = JSON.parse(fileContent);
} catch(err) {
console.error("The file for subtypes is invalid and can't be read:");
console.error(err);
expect(err).toBeUndefined();
}

expect(isObject(schema)).toBeTruthy();
expect(isObject(schema.definitions)).toBeTruthy();

// lint: Check whether the file is correctly JSON formatted
expect(normalizeString(JSON.stringify(schema, null, 4))).toEqual(normalizeString(fileContent.toString()));

// Is JSON Schema valid?
checkJsonSchema(await getAjv(), schema);

// is everything dereferencable?
let subtypes = await $RefParser.dereference(schema, { dereference: { circular: "ignore" } });
expect(isObject(subtypes)).toBeTruthy();
});
54 changes: 54 additions & 0 deletions tests/subtypes-schemas.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
const $RefParser = require("@apidevtools/json-schema-ref-parser");
const { checkDescription, checkSpelling, isObject } = require('./testHelpers');

// I'd like to run the tests for each subtype individually instead of in a loop,
// but jest doesn't support that, so you need to figure out yourself what is broken.
// The console.log in afterAll ensures we have a hint of which process was checked last

// Load and dereference schemas
let subtypes = {};
let lastTest = null;
let testsCompleted = 0;
beforeAll(async () => {
subtypes = await $RefParser.dereference('../meta/subtype-schemas.json', { dereference: { circular: "ignore" } });
return subtypes;
});

afterAll(async () => {
if (testsCompleted != Object.keys(subtypes.definitions).length) {
console.log('The schema the test has likely failed for: ' + lastTest);
}
});

test("Schemas in subtype-schemas.json", () => {
// Each schema must contain at least a type, subtype, title and description
for(let name in subtypes.definitions) {
let schema = subtypes.definitions[name];
lastTest = name;

// Schema is object
expect(isObject(schema)).toBeTruthy();

// Type is array with an element or a stirng
expect((Array.isArray(schema.type) && schema.type.length > 0) || typeof schema.type === 'string').toBeTruthy();

// Subtype is a string
expect(typeof schema.subtype === 'string').toBeTruthy();

// Check title
expect(typeof schema.title === 'string').toBeTruthy();
// lint: Summary should be short
expect(schema.title.length).toBeLessThan(60);
// lint: Summary should not end with a dot
expect(/[^\.]$/.test(schema.title)).toBeTruthy();
checkSpelling(schema.title, schema);

// Check description
expect(typeof schema.description).toBe('string');
// lint: Description should be longer than a summary
expect(schema.description.length).toBeGreaterThan(60);
checkDescription(schema.description, schema);

testsCompleted++;
}
});
7 changes: 6 additions & 1 deletion tests/testHelpers.js
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,10 @@ async function getAjv() {
return jsv;
}

function isObject(obj) {
return (typeof obj === 'object' && obj === Object(obj) && !Array.isArray(obj));
}

function normalizeString(str) {
return str.replace(/\r\n|\r|\n/g, "\n").trim();
}
Expand Down Expand Up @@ -214,5 +218,6 @@ module.exports = {
checkSpelling,
checkJsonSchema,
checkSchemaRecursive,
prepareSchema
prepareSchema,
isObject
};

0 comments on commit 3faf89d

Please sign in to comment.