Release v23.7.1 · Sage-Bionetworks/schematic

Release Notes

Allow users to set strictness of google sheet/Excel regex validation via the Schematic API. Previously the default schematic configuration of strict_validation=true did not allow a user to proceed if an incorrect value was entered. By exposing the strict_validation option in the manifest generation api, users are now able to select True or False. strict_validation=false will allow users to enter an incorrect value, but be served a warning. In either case, incorrect entries will not pass manifest validation.
Breaking change for 23.7.1 - Simplified Schematic configuration - this will require config changes as of 23.7.1
- Please adjust your config file following the documentation example and Readme:
- https://github.com/Sage-Bionetworks/schematic/blob/develop/config_example.yml
- Readme (see Configure config.yml File section)
Optimized the performance in table upsert backwards compatibility by Adding functionality to the synapseStorage object to make calls to the synapse REST API by way of the functionality exposed in the synapsePythonClient.

Addressed regex search validation limitations in Excel and updated documentation on use.
Ensure that entity annotations are always compliant with Synapse regardless of column name format to avoid causing an error.
Addressed bugs to ensure that submission and annotation are occurring as expected

Code doesn't escape the 2nd law of thermodynamics. We put energy into refactoring handling of validation rules and interactions with Synapse (so that adding features and avoiding bugs is easier later); catching errors and exceptions more robustly and specifically (so that users and clients know what's causing a problem and can handle, report, or fix it more effectively); improving coverage of automated testing (so that we reduce the likelihood of letting bugs in released versions of schematic).

For more details on specific changes, please refer to the changelog below.

Full Changelog: v23.6.3...v23.7.1

Replaced synapse API calls with synapse Python client call by @linglp in #1235
Feature fds 273 coniguration by @andrewelamb in #1219
Expose strict_validation option for manifest/generate endpoint by @mialy-defelice in #1253
Optimize performance in table upsert backwards compatibility scenarios by @GiaJordan in #1229
Fix df_utils/load_df so it more accurately captures integers to prevent Regex Errors. by @mialy-defelice in #1240
Use FAIR Data service desk for issues by @afwillia in #1257
Update Black to 2023 version: 23.7.0 by @GiaJordan in #1262
Update mypy to latest version by @GiaJordan in #1263
Updated cli help text for parameter json_schema by @linglp in #1264
Modified Nginx parameters to fix submission issue by @linglp in #1255
modify tag by @linglp in #1251
BugFix: Fixed an issue where files on synapse were not being annotated correctly by @GiaJordan in #1254
[bug fix] Fixed errors when calling get_empty_manifest by @linglp in #1260
[bug fix] Updated parameters being used when generating existing manifests in test_api.py by @linglp in #1261
Update schematic_db minimum version to v0.0.29 by @GiaJordan in #1266
Fix Google Sheet column info mismatch FDS-675 by @mialy-defelice in #1265
Only add validation rules to google sheets by @mialy-defelice in #1252
Make all entity annotations comply with synapse FDS-481 FDS-726 by @GiaJordan in #1267
Schematic 23.7.1 - FDS-728 by @linglp in #1271

Full Changelog: v23.6.3...v23.7.1