Schematic version compatibility and schema data-model version

[leexgh]

Schematic

We use SchemaExplorer in script get_syn_data.py, SchemaExplorer is removed / refactored in new version of Schematic, v23.12.1. is the last working version contains SchemaExplorer.

schema data-model

The latest HTAN.model.jsonld (as of 2024/07/25) can't pass the validation, one workaround is to download this file to local, edit to fix the StripeCalling validation rule, replace this URL to local path, and rerun the script.

Debug result

Schematic version	HTAN.model.jsonld version	error message	possible explanation
v24.7.1	4ab87b2 (latest as of 2024/07/25)	Traceback (most recent call last): File "/Users/lix2/Documents/GitHub/htan-portal/data/get_syn_data.py", line 18, in from schematic.schemas.explorer import SchemaExplorer ModuleNotFoundError: No module named 'schematic.schemas.explorer'	SchemaExplorer is removed/refactored in Schematic
v23.12.1	4ab87b2 (latest as of 2024/07/25)	raise ValidationError( jsonschema.exceptions.ValidationError: The json_schema has an error in the validation_rules set for attribute StripeCalling. Validation failed with the following errors: [['NA', 'StripeCalling', 'The json_schema, has an error in the validation rule for the attribute: StripeCalling, the provided validation rules (-?\d+) is not a valid rule. Please check spelling.', 'Not a Rule']]	v23.12.1 Schematic has SchemaExplorer so don't see the module not found error. StripeCalling's validation rule in HTAN.model.jsonld is not correct, should be " regex search -?\\d+“ instead of "-?\\d+"
v23.12.1	57da854(older version, 2024/05/30, no StripeCalling validation rule)	Success! (30+ missing types from schema)	This is an older version schema so missing types from schema is expected, and also confirm using older version of schema could work
v23.12.1	4ab87b2 (latest as of 2024/07/25, but change StripeCalling's validation rule locally)	Success! (only one type is missing - BrainCancerTier3 does not exist schema )	This is the way I make new data release PR