Skip to content

Releases: metaschema-framework/metaschema-java

v2.1.0

26 Nov 18:26
@david-waltermire david-waltermire
v2.1.0
462da0c
Compare
Choose a tag to compare
v2.1.0 Latest
Latest

This minor release incorporates a bunch of bug fixes, new features, and code refactoring that will improve the stability of runtimes, and the maintainability of the code and use of the APIs.

What's New

  • The following new Metapath functions were added:

  • The time and time-with-timezone data types were added to support the fn:current-date Metapath function. (#265)

  • Support for the following Metapath expressions was added:

  • Expanded the Metapath ANTR grammar to include kind tests. (#136)

    • Completed support for flag, field, assembly, and document kind tests.
    • Added support for IArrayItem contents IItemType matching.

  • The internal type system was greatly improved to support these new Metapath expressions. This system is more robust and lays the groundwork for "instance of", "castable", other type related tests, and dynamic function declarations. (#136, #264)

    • All IItem and ICollectionValue types now have an atomize() method, which provides the prior functionality of the similar names method in FnData,
    • Cleaned up use of getAtomicItem() methods to ensure more consistent use.
    • Added IItem.toSignature() and ICollectionValue.toSignature(), which provides the type signature of the object based on its type. The toString() method on these implementations will also provide the same signature to aid in debugging.
    • Refactored QName use to use an new IEnhancedQName that supports caching qualified names using a numeric index. Name resolution now handled by the StaticContext. Switch flag and model name resolution from IModule to IModule.getStaticContext().
    • Replaced QName based lookups with integer index-based lookups. Use of integer-based indexes should reduce memory usage and enables more efficient integer-based lookups in place of string-based matching.
    • Added support for sequence type matching on sequences.
    • Refactored data type service to handle abstract types that have no associated adapter so that all data types can be discovered using the service.
    • Refactored data and date/time data types to have the ambiguous data type interfaces be the parent of the non-ambiguous data types. This aligns better with the data type naming and the semantics of how they work.
    • Refactored data type and atomic item implementations to move casting operations to the atomic item implementations. This creates a cleaner isolation between the type adapters and Metapath items. (#245)
    • Improved date and date-time adapter unit testing to check for ambiguity and resulting time value.

  • Other refactoring, includes: (#136)

    • Cleaned up all IGroupable implementations to simplify code and reduce methods.
    • Refactored the implementations supporting model container generation, by reducing the number of implementations and implementing a builder pattern to store container information while the container is being built. This enabled cleanup of some duplicate code identified by CPD in XmlFlagContainerSupport and other classes.
    • Removed a bunch of unnecessary interfaces and unwound some significant spaghetti code.
    • Moved the sequence collector toSequencefrom ISequence to CustomCollectors.
    • Added a location hint to ISource which will provide the URI or class for the Metaschema module. Cleaned up source information to provide more context in exceptions. Improved some exceptions using this information.
    • Refactored exception handling in StaticContext lookup methods.
    • Added a bunch of missing Javadocs.
    • Added many missing Javadocs. Adjusted many existing Javadocs to provide more details. (#245)
    • Removed some unused constants.
    • Performed some light refactoring to remove unused code and to improve overall code readability to make maintenance easier.
    • Replaced Paths.get("") with Paths.get(System.getProperty(user.dir)) to be more explicit.
    • Improved value creation methods in date and date/time item classes.
    • Refactored the result type used in MetapathExpression to avoid the need for a case statement. This should result in cleaner code and better performance overall.
    • Removed some commented out code.
    • Updated some inconsistent Log4j2 configurations to be consistent with other configurations.
    • Updated build and unit testing dependencies through parent POM.
    • Improved parser errors and warnings to include the resource the error was found in.

  • Unit test improvements:

    • Added some unit test assertions to ensure HTML to Markdown conversion is tested more completely. (#245)
    • Significantly improved data and date/time parsing test vectors.

  • Updated the Metaschema submodule to latest in metaschema-framework/metaschema develop.

Bug Fixes

  • Resolved #259 by delaying registering module in validation commands to just before constraint validation to ensure schemas are generated directly from the loaded module and not the compiled version. (#263)
  • Fixed a bug related to how sequence transitions from IArrayItem and IMapItem were handled, where they weren't unwrapped properly. (#136)
    • Split ICollectionValue.asSequence() into two methods: 1) toSequence(), which has the same behavior of generating a sequence containing the value. 2) contentsAsSequence() which will return the item's contents as a ISequence if the item is multivalued (i.e., IArrayItem, IMapItem), or the item otherwise.
  • Resolved @258 by disallowing partial metapath parsing, which allowed invalid metapaths to be used if the first part was seen as valid. **Note: This may result in some erroneous Metapaths now being caught, where previously they were allowed. Double checked OSCAL internal constraints to make sure that this wasn't the case there. (#260)
  • Fixed an issue that prevented constraints to be applied to the built-in Metaschema module module.
  • Also fixed an issue that prevented newer binding matchers from overriding previous binding matchers where they have the same element names.

Breaking Changes

There are some minimal breaking changes in this release that are limited to publicly visible implementation classes focused around data type, Metapath items, Metapath function declaration, the Metapath StaticContext, and some internal model classes. These changes will not affect the majority of applications using this library, unless the library is providing custom data types or Metapath functions. liboscal-java was updated based on these changes, and provides a good example of the minimal scope of these changes.

What's Changed

Full Changelog: v2.0.1...v2.1.0

Contributors

258, david-waltermire, and dependabot
Assets 2
Loading

v2.0.1

09 Nov 21:35
@david-waltermire david-waltermire
v2.0.1
964b845
Compare
Choose a tag to compare
v2.0.1

This patch release fixes a few bugs that affect the output of tooling using this library.

What's Changed

Full Changelog: v2.0.0...v2.0.1

Contributors

david-waltermire
Assets 2
Loading

v2.0.0

03 Nov 17:36
@david-waltermire david-waltermire
v2.0.0
0859d4a
Compare
Choose a tag to compare
v2.0.0

This major release incorporates a bunch of bug fixes, new features, and code refactoring that will improve the stability of runtimes, and the maintainability of the code and use of the APIs.

What's New

  • In #189, the metaschema-cli convert command was added which allows for the conversion between supported formats for any content that conforms to a Metaschema module.

    usage: metaschema-cli convert --to=FORMAT -m=FILE_OR_URL [<options>] <source-file-or-URL> [<destination-file>]
 -h,--help               display this help message
 -m <FILE_OR_URL>        metaschema resource
    --no-color           do not colorize output
    --overwrite          overwrite the destination if it exists
 -q,--quiet              minimize output to include only errors
    --show-stack-trace   display the stack trace associated with an error
    --to=<FORMAT>        convert to format: xml, json, or yaml
    --version            display the application version

  • In #181, the logging produced by validation commands has been enhanced to also output the constraint id that the log message pertains to.

  • The following new Metapath functions were added:

    • fn:lower-case (#203)
    • fn:string-length() (#193)
    • fn:substring-after (#195)
    • fn:substring-before (#194)
    • fn:upper-case (#203)

  • let statements now fully support exposing the remark. Previously, this support was lacking for the pre-generated class bindings
    for the Metaschema.

  • The flag, following, following-sibling, preceding, and preceding-sibling axes are now implemented. See #229.

  • In allowed-values constraints, deprecated versions are now properly exposed in the related APIs.

  • All constraint types, except for allowed-values now support custom messages similar to how expect previously worked. See #218.

  • Runtime errors that occur while validation content using constraints sometimes caused the validation to exit prematurely before reporting validation errors. PR #232 now ensures that validation continues and these runtime issues are reported as validation errors.

  • Definition and instance properties are now fully supported in the class binding APIs.

  • The metaschema-cli eval command now outputs the value for flag and field nodes in addition to the path to the node. See #228.

  • Due to the differences between how null values are handled in XML and JSON/YAML, the parser will now report warnings when a null value is found. See #227 for more information on this behavior.

Bug Fixes

  • PR #188 addressed #184 by ensuring that only one copy of a constraint is added at a given location within a model. This issues was caused when an external constraint applied to multiple locations within a model and that model construct was reused in one or more of those locations. A copy of the constraint was added for each time the constraint target matches one of those locations.
  • PR #187 addressed the case where a search path (//somenode). In such cases, the matched nodes were returned multiple times. This fix ensure that the nodes returned are unique.
  • PR #214 fixes the case where if the fn:doc-available Metapath function is called with an absolute URI, the function was trying to still resolve it. This would cause an error when the static context doesn't have a baseUri set. The behavior has been changed to only try resolution when the URI is not absolute.

Breaking Changes

  • Some implementation classes and interfaces haven been moved or renamed. The following is a summary of the changes.

    In package gov.nist.secauto.metaschema:

    • core.resource.AbstractResourceResolver → core.model.AbstractResourceResolver
    • core.model.xml.ExternalConstraintsModulePostProcessor → core.model.constraint.ExternalConstraintsModulePostProcessor
    • core.model.constraint.ISource → core.model.ISource
    • databind.model.binding.metaschema.* → databind.model.metaschema.binding.*
    • core/util/DefaultDiagramNode → core/model/util/DefaultDiagramNode
    • core/util/IDiagramNode → core/model/util/IDiagramNode
    • core/util/IDiagramNodeVisitor → core/model/util/IDiagramNodeVisitor
    • core/util/MermaidErDiagramGenerator → core/model/util/MermaidErDiagramGenerator

  • The interface gov.nist.secauto.metaschema.databind.IBindingContext has been significantly changes. You can no longer get a global shared instance. Instead, you can get a new instance using the newInstance() method, or a builder instance using the builder() method. These changes were made in #211.

    By default, new instances do not support dynamically generating and compiling class bindings. This can be enabled by calling the compilePath(Path) using the builder instance.

What's Changed

Read more

Contributors

david-waltermire, dependabot, and aj-stein-gsa
Assets 2
Loading

v1.2.0

08 Oct 22:46
@david-waltermire david-waltermire
v1.2.0
46df8d8
Compare
Choose a tag to compare
v1.2.0

Changelog

This minor release adds the following new features:

Bug Fixes

  • In prior releases, validation results produced in the Static Analysis Results Interchange Format (SARIF) were not valid according to the SARIF JSON schema. This release fixes these issues. (#137)
  • Property values were erroniously limited to token values, instead of string values (#177)

What's Changed

New Contributors

Full Changelog: v1.1.0...v1.2.0

Contributors

david-waltermire, dependabot, and aj-stein-gsa
Assets 2
Loading

v1.1.0

14 Sep 13:11
@david-waltermire david-waltermire
v1.1.0
874ad2d
Compare
Choose a tag to compare
v1.1.0

Changelog

This minor release adds the following new features.

  • Adds a new CLI command that supports arbitrary Metapath evaluation in the command line interface (CLI).

    The command is metaschema-cli metapath eval.

    usage: metaschema-cli metapath eval -m=FILE_OR_URL --expression=EXPRESSION [<options>]
 -e,--expression=<EXPRESSION>   Metapath expression to execute
 -h,--help                      display this help message
 -i <FILE_OR_URL>               Metaschema content instance resource
 -m <FILE_OR_URL>               metaschema resource
    --no-color                  do not colorize output
 -q,--quiet                     minimize output to include only errors
    --show-stack-trace          display the stack trace associated with an error
    --version                   display the application version

  • Adds a new CLI command for generating a Mermaid diagram for a Metaschema module.

    usage: metaschema-cli generate-diagram [<options>] <metaschema-module-file-or-URL> [<destination-diagram-file>]
 -h,--help               display this help message
    --no-color           do not colorize output
    --overwrite          overwrite the destination if it exists
 -q,--quiet              minimize output to include only errors
    --show-stack-trace   display the stack trace associated with an error
    --version            display the application version

  • In the metaschema-cli validate-content command, you can now disable schema or constraint validation using arguments.

    --disable-constraint-validation   do not perform constraint validation
--disable-schema-validation       do not perform schema validation

  • Removed a warning around improperly contextualized Metapaths, that was identifying cases in external constraints where sub contexts and constraint targets use an absolute path expression. This feature creates warnings in the CLI that add noise to the results without adding value.

    This change was checked against the OSCAL modules to ensure backwards compatibility and no impact to existing validations.

What's Changed

Full Changelog: v1.0.2...v1.1.0

Contributors

david-waltermire and dependabot
Assets 2
Loading

v1.0.2

19 Aug 23:02
@david-waltermire david-waltermire
v1.0.2
76fe3c9
Compare
Choose a tag to compare
v1.0.2

What's Changed

Full Changelog: v1.0.1...v1.0.2

Contributors

david-waltermire
Assets 2
Loading

v1.0.1

04 Aug 05:34
@david-waltermire david-waltermire
v1.0.1
87dd858
Compare
Choose a tag to compare
v1.0.1

This patch release does not contain any functional code changes. It updates the website with a new format.

What's Changed

  • Bump actions/upload-artifact from 4.3.3 to 4.3.5 by @dependabot in #67
  • Bump org.apache.commons:commons-text from 1.11.0 to 1.12.0 by @dependabot in #63
  • Bump commons-io:commons-io from 2.15.1 to 2.16.1 by @dependabot in #66
  • Bump org.codehaus.plexus:plexus-utils from 4.0.0 to 4.0.1 by @dependabot in #65

Full Changelog: v1.0.0...v1.0.1

Contributors

dependabot
Assets 2
Loading

v1.0.0

04 Aug 05:34
@david-waltermire david-waltermire
v1.0.0
95b24a3
Compare
Choose a tag to compare
v1.0.0

Changelog

This release contains unreleased commits from the NIST repository and changes that were requested, but not merged at the time of this release, including usnistgov/metaschema-java#374, usnistgov/metaschema-java#368, usnistgov/metaschema-java#367, usnistgov/metaschema-java#365, usnistgov/metaschema-java#363, usnistgov/metaschema-java#373, and usnistgov/metaschema#659. The commits for these PRs are also merged here.

This release includes:

  • significant cleanup of the code base, providing for easier long-term maintenance
  • a large number of new Metapath features and functions
  • robust support for external constraints that allows for further restrictions over an existing Metaschema module when validating content
  • many bug fixes that improve the reliability of this project
  • the ability to produce validation results in the Static Analysis Results Interchange Format (SARIF), which allows for integration with tools implementing this standard.

General

  • Added and improved Javadoc comments across all modules, with a focus on full documenting the core API.
  • Cleaned up many compile, code style, SpotBugs, CPD, and PMD warnings and errors.
  • Cleaned up orphaned code
  • Switched some tests using resources in the test classpath to use file resources to make testing more stable.
  • Refactored model interfaces in core and databind to reduce code duplication, to simplify interface heirachies, and to share more implementation code. This resulted in the removal of dozens of interfaces, which were no longer needed.
  • Cleaned up some accessor method names to be more consistent with their return types.
  • Added support for defining configuration feature names to better support debugging.
  • Cleaned up stream closure to ensure that streams are properly closed.
  • Added methods to handle making URIs relative to another URI.
  • Some work towards improving NullPointerExceptions to provide more context when missing data is the cause.

metaschema-core

Metaschema Module Model

  • Implemented choice group support based on usnistgov/metaschema#228, which provides a means to allow sequenced collections of objects of different types.
  • Cleaned up and documented loader interfaces, to include IDocumentLoader.
  • Refactored XML model parsing. Moved XML model classes to be hidden in the implementation.
  • Added support for QName in Metaschema data types, which are used in function signatures now instead of the Java class name.
  • Fixed XML choice model parsing to ensure instances are also added to the model instance list.
  • Refactored core model classes and support for collections
    • Relocated many class files and cleaned up many unneeded methods.
    • Reworked definition and instance model classes to reduce extra interfaces and to simplify and align implementations.
    • Added support for flag and field instances to provide a different default value than the definition.
    • Refactored instance class hierarchy to support grouped and ungrouped model instances, with minimal implementation.
      • Renamed model interfaces for greater consistency.
      • Refactored core model interfaces to reduce inheritance conflicts.
      • Adjusted the module interfaces to promote a cleaner, non-parameterized API.
    • Renamed many instance implementation classes to ensure consistent use of terminology. Prior the terms "property" and "instance" were used interchangeably, which could cause some confusion.
    • Refactored IBindingContext to move supporting interfaces to be child interfaces.
      • Removed the need for instances of IDataTypeHandler. Now IClassDataTypeHandler functionality is provided by IFeatureComplexItemValueHandler and JavaTypeAdapterDataTypeHandler is provided by IFeatureScalarItemValueHandler.
      • Refactored bound class hierarchy to be more consistent with needed methods.
      • Moved Java field handling to IFeatureJavaField from IBoundNamedModelInstance
      • Moved JSON key and item handling to IBoundModelInstance.
      • Reworked XML and JSON reading and writing.
      • Added IBoundChoiceGroupInstance interface for choice groups.
    • Cleaned up IModelPropertyInfo and implementations, relocating supporting classes as child classes.
      • Implemented a IModelInstance item parsing as IModelPropertyInfo.IReadHandler, allowing all item parsing to be implemented in the format-specific parsing classes, i.e. MetaschemaJsonReader, MetaschemaXmlReader.
      • Cleaned up IModelPropertyInfo readItems methods, which now return the parsed value. Eliminated the need for IPropertyCollector instances.
    • Refactored constraint ILet statement construction to use static factory methods.
    • Cleaned up JSON value key naming code.
    • Fixed default value handling which wasn't producing empty collection for unparsed model instances. Cleaned up getDefaultValue and getEffectiveDefaultValue methods.
    • Refactored JSON key handling to determine the JSON key on a per item basis. This is needed for ChoiceGroups.
    • Refactored container handling code to reuse the core model implementation where possible.
      • Refactored group-as namespace handling to allow for no namespace situations. Implemented getEffectiveGroupAsNamespace to handle the default case of using the module namespace where no group-as support is provided.

Metapath Interpreter

  • Added support for more XPath 3.1 syntax in Metapath:

    • Adjusted Metapath syntax to add support for for, if, range, simple map !, and extended QNames.
    • Added support for Metapath if expressions: if (expr) then (expr) else (expr).
    • Added support for Metapath range expressions: intValue to intValue.
    • Added support for Metapath quantified expressions: (some | every) $VarName in Expr1 (, $VarName in ExprN)* satisfies ExprS.

  • Added the following Metapath functions:

    • abs
    • avg
    • ceiling
    • concat
    • count
    • document-available
    • empty
    • head
    • insert-before
    • min
    • max
    • sum
    • tail
    • remove
    • reverse
    • array:get
    • array:put
    • array:append
    • array:subarray
    • array:remove
    • array:insert-before
    • array:join
    • array:head
    • array:tail
    • array:reverse
    • array:flatten
    • map:get
    • map:merge
    • map:entry
    • map:size
    • map:keys
    • map:contains
    • map:find
    • map:put
    • map:remove
    • meta:model which gets a nodeset based on the underlying Metaschema module model for the node.

  • Fixed defects in the following functions:

    • data
    • false
    • true
    • starts-with which was checking for contains not startsWith.

  • Added support for sub-DynamicContexts supporting variable definition in let and quantified expression statements.

  • Added support for map construction.

    • Added support for maps in function call, postfix and unary lookups.
    • Added missing characteristics on a number of existing Metapath functions.

  • Added support for Metapath arrays

    • Completed code to support Metapath array construction.
    • Implemented array postfix and unary lookups.

  • Changed ISequence to implement the List interface to allow sequences to better participate in collection-oriented Java methods.

  • Cleaned up ISequence implementations, moving getFirstItem to be a method member. Added a utility function to convert a sequence into an array member.

  • Refactored comparison methods into a new ComparisonFunctions class.

  • Reorganized and improved item implementations to better group interfaces and implementation classes and to be more robust.

    • Refactored atomic item interfaces and implementations.
    • Added support for casting an IItem of IAnyAtomicItem type to the type of another item using the castAsType method. This is useful for functions that require the return type to match the argument's type.
    • Added support for compareTo for all atomic items.
    • Added equals and hashCode to all item types.
    • Renamed FlagDefinitionNodeItemImpl to FlagGlobalDefinitionNodeItemImpl to better harmonize with sibling field and assembly implementations.
    • Added toString methods to some items to improve debugging and testing.
    • Refactored IDocumentNodeItem and IModuleNodeItem to better align the features of each. A document node item not exposes the root assembly, while a module node item exposes the global definitions and module namespace.
    • Ensured proper handling of Metapath errors during validation. Resolves usnistgov/oscal-cli#292

  • To improve maintenance of Metapath function test vectors, inline compilation of Metapaths are now used, making tests more readable while we working on usnistgov/metaschema#239.

  • Fixed a bug causing Metapath lexer errors to be silently ignored.

  • Reorganized and refactored the Metapath compact syntax tree and ANTLR-based abstract syntax tree implementation classes.

  • Fixed bug causing empty sequences to not be produced by the Metapath expression: ().

  • Cleaned up static and dynamic Metapath exceptions. Refactored StaticContext generation to ensure proper use of model-based methods.

  • Adjusted namespace of Metapath functions to http://csrc.nist.gov/ns/metaschema/metapath-functions.

  • Improved function testing to lookup and call the function. This will more fully exercise the function call in JUnit tests.

  • Fixed a defect in opNumericDivide that caused improper handling of integer division.

  • Adjusted the document node item to ensure it returns the root node when getValue is called. This fixes usnistgov/oscal-cli#216.

  • Added support for Expanded QNames

  • Fixed and tested mod expression handling to properly return an integer result

  • Fixed function chain arrow operator processing

    Made => operator eqname only in A...

Read more

Contributors

david-waltermire, dependabot, and 2 other contributors
Assets 2
Loading