Add spec violation rules #141

richfab · 2023-10-24T21:16:28Z

This PR follows #106.

Brings all the code from Additional errors #106.
Removes the validation rules that are not in the spec, as agreed in Additional errors #106 (comment). The code removed can be found in Additional errors #139.
Collapses errors and warnings in the UI (they were expanded by default in Additional errors #106)

Before	After

Feed used for screenshots: https://acoruna.publicbikesystem.net/customer/gbfs/v2/gbfs.json

Signed-off-by: Tom <[email protected]>

Signed-off-by: Pierrick <[email protected]>

Signed-off-by: Tom <[email protected]>

netlify · 2023-10-24T21:16:33Z

✅ Deploy Preview for gbfs-validator ready!

Name	Link
🔨 Latest commit	`d271ba8`
🔍 Latest deploy log	https://app.netlify.com/sites/gbfs-validator/deploys/6544130a08b2b400080fbb26
😎 Deploy Preview	https://deploy-preview-141--gbfs-validator.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

testower

I think a few non-trivial issues remain unresolved:

Taxonomy of validation result

Do we need to differentiate between schema and non-schema errors?

Multiple patterns in use for validation

The are several patterns in use:

original schema
patched schemas ("partials")
imperative rules (in JavaScript),
and some hybrid of the latter two it seems (in validation of additionalProperties).

I think this is problematic, and it should be clarified what we prefer, and the codebase needs to be clearer.

I think one benefit of using schemas and patched schemas only is that the validation itself is done by ajv and all the results is an output of ajv. It has been argued that these are harder to read than the imperative style, but after reading this PR I'm not 100% convinced.

Another problem with the imperative style is the unclear connection to the version(s) that the rule applies to. In the schema patching pattern it is explicit, whereas here you have to read the code (at least if it's not documented) - and mistakes are harder to spot.

Misc

The rest of the review is mostly technical and cosmetic.

One issue that stood out is inappropriate usage of Array methods, especially map where forEach would be better, and some others.

Some files are getting quite hard to read, and are in dire need of refactoring.

Some confusing folder structure introduced in test folder.

gbfs-validator/nonSchemaValidation/ttl.js

testower · 2023-10-25T11:33:52Z

gbfs-validator/__test__/fixtures/conditionnal_no_vehicle_type_id.js

@@ -1,5 +1,7 @@
 const fastify = require('fastify')

+const last_updated_fresh = Math.floor(Date.now() / 1000) - 30


It is good practice to not use real clocks in tests. It makes tests more reliable and readable. Jest has as of v27 the ability to do this neatly: https://jestjs.io/docs/jest-object#jestsetsystemtimenow-number--date

This is a great idea, but this is beyond my knowledge unfortunately (even after reading the docs). Anyone, feel free to suggest an implementation. Thanks!

testower · 2023-10-25T11:35:27Z

gbfs-validator/__test__/gbfs.v3.0-RC.test.js

    })
  })

-  return errors
+  return { errors, nonSchemaErrors, warnings }


I still think we need to review the taxonomy here. Why do we differentiate between errors and nonSchemaErrors? Would we not simply just have errors and warnings (where warnings are SHOULD violations)?

I tried to combine errors and nonSchemaErrors but was limited by my technical skills. Anyone, feel free to suggest an implementation.

gbfs-validator/__test__/fixtures/v3.0-RC/exaustive.js

testower · 2023-10-25T11:45:43Z

gbfs-validator/__test__/gbfs.v3.0-RC.test.js

+  beforeAll(async () => {
+    const { MockRequests } = require('./fixtures/v3.0-RC/exaustive')
+
+    class InvalidMockRequests extends MockRequests {


I don't really understand what is going on here? Why not just create a new fixture for this test?

This is beyond my technical skills. Anyone, feel free to suggest an implementation.

gbfs-validator/nonSchemaValidation/check_geofencing_zones.js

testower · 2023-10-25T19:17:32Z

gbfs-validator/nonSchemaValidation/translations.js

+  }
+
+  for (const lang of languages) {
+    const filtered = data.filter((a) => a.language === lang)


Array#some is more efficient here (and condition can be inlined)

This is beyond my technical knowledge. Anyone, feel free to suggest an implementation. Thanks!

gbfs-validator/versions/v1.0.js

testower · 2023-10-25T19:54:48Z

One thing I should add in my quest for schema-centric validation is of course that the SHOULD/warning validations are hard to implement that way...

richfab · 2023-11-02T21:26:44Z

@testower I'm done applying the quick fixes from your suggestions. The other comments are beyond my technical knowledge. I will sync with @davidgamez about next steps. Thanks!

richfab · 2023-11-13T16:27:21Z

Thank you @tdelmas for the initial PR (#106) and @testower for the thorough PR review.

Based on the conversation following #153 (comment), we want to try doing the validation of these new rules by generating new schemas via a builder or a template instead of using imperative rules (eg: JavaScript).

For this reason, I believe this PR can be closed in favor of the following issues:

Thank you!

tdelmas and others added 10 commits October 24, 2023 11:50

Custom validations

92c46af

Signed-off-by: Tom <[email protected]>

Better display & error counts

50dfba6

Signed-off-by: Tom <[email protected]>

Regroup all errors on front

12c640a

Signed-off-by: Pierrick <[email protected]>

Rename otherValidation to nonSchemaValidation

9cc5938

Signed-off-by: Pierrick <[email protected]>

Add duplicate_vehicle_type_id error

2170404

Signed-off-by: Pierrick <[email protected]>

Fix typos

f9dfc5a

Signed-off-by: Tom <[email protected]>

Fix when errors is false but not nonSchemaErrors

deae327

Signed-off-by: Tom <[email protected]>

Most review suggestions

bb49a1c

Signed-off-by: Tom <[email protected]>

Removes the rules that are not in the spec

1435e7f

Collapse errors and warnings

bad9010

richfab requested review from PierrickP and testower October 24, 2023 21:16

richfab added the help wanted Extra attention is needed label Oct 24, 2023

richfab mentioned this pull request Oct 24, 2023

Additional errors #106

Closed

testower suggested changes Oct 25, 2023

View reviewed changes

richfab added 13 commits November 1, 2023 14:54

Provide the actual value of last_updated

598e14f

Fix typo in exhaustive

352b25b

Improve test message

00028ff

Return the result of hasErrors and hasWarnings

a3e498e

Specify from which version the rules apply

86cd938

Use forEach in loops instead of map

8f4ba14

Fix line starting with semicolon

3054d26

Use Array#some

7013a67

Take metadata out of error message for consistency

1e0569d

Typo in multipolygon

e747ccb

Variable name for coordinates of a multipolygon

1844e72

Variable name polygonCoordinates

cf2bfae

Rename variable o to rules

b4a2832

Undo Use Array#some because the tests failed

d271ba8

richfab mentioned this pull request Nov 10, 2023

Object dependency relationships validation #153

Open

richfab closed this Nov 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add spec violation rules #141

Add spec violation rules #141

richfab commented Oct 24, 2023 •

edited

Loading

netlify bot commented Oct 24, 2023 •

edited

Loading

testower left a comment •

edited

Loading

testower Oct 25, 2023

richfab Nov 1, 2023

testower Oct 25, 2023

richfab Nov 1, 2023

testower Oct 25, 2023

richfab Nov 1, 2023

testower Oct 25, 2023

richfab Nov 2, 2023

testower commented Oct 25, 2023

richfab commented Nov 2, 2023

richfab commented Nov 13, 2023

		@@ -1,5 +1,7 @@
		const fastify = require('fastify')

		const last_updated_fresh = Math.floor(Date.now() / 1000) - 30

Add spec violation rules #141

Add spec violation rules #141

Conversation

richfab commented Oct 24, 2023 • edited Loading

netlify bot commented Oct 24, 2023 • edited Loading

✅ Deploy Preview for gbfs-validator ready!

testower left a comment • edited Loading

Choose a reason for hiding this comment

Taxonomy of validation result

Multiple patterns in use for validation

Misc

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

testower commented Oct 25, 2023

richfab commented Nov 2, 2023

richfab commented Nov 13, 2023

richfab commented Oct 24, 2023 •

edited

Loading

netlify bot commented Oct 24, 2023 •

edited

Loading

testower left a comment •

edited

Loading