Code

IS-12 Test Suites#810

Code

Open

jonathan-r-thorpe wants to merge 191 commits intomasterAMWA-TV/nmos-testing:masterfrom is-12AMWA-TV/nmos-testing:is-12Copy head branch name to clipboard

+5,237-41

Commits

Commits on May 9, 2023

• [IS-12] Initial Commit for IS-12 Test Suite (#800)

  Show description for ec40a1d
  jonathan-r-thorpe

  authored
  ec40a1d

Commits on Jun 21, 2023

• [IS-12] Check Root Block and ClassManager (#814)

  Show description for 419ef85
  jonathan-r-thorpe

  authored
  419ef85

Commits on Jul 18, 2023

• [IS-12] Add auto tests for validating MS-05 standard class and datatype definitions (#819)

  Show description for c033475
  

  3 peopleauthored
  c033475

Commits on Jul 20, 2023

• Merge pull request #812 from AMWA-TV/zeroconf-marmoset

  Show description for dadf52b
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  dadf52b

Commits on Aug 4, 2023

• [IS 12] Add Get/Set/Add/Remove/Find methods (#822)

  Show description for e36e6cc
  jonathan-r-thorpe

  authored
  e36e6cc

Commits on Aug 8, 2023

• [IS-12] Specify explicit URL path for WebSocket endpoint (#827)

  Show description for f841bb7
  jonathan-r-thorpe

  authored
  f841bb7

Commits on Aug 10, 2023

• Check response from Find methods is an array

  jonathan-r-thorpe

  committed
  f8c63b6

Commits on Aug 23, 2023

• Handle errors where control classes are not advertised by Class Manager

  jonathan-r-thorpe

  committed
  c28ee82

Commits on Aug 24, 2023

• Fixed detection of authority keys in non-standard classes

  jonathan-r-thorpe

  committed
  d22005d

• Handle missing device model properties

  jonathan-r-thorpe

  committed
  8073aa2

• Defend against missing properties or datatype definitions

  jonathan-r-thorpe

  committed
  2bbd542

Commits on Oct 6, 2023

• Improved error handling,

  jonathan-r-thorpe

  committed
  fa810fa

• Bump Node 12.x to Node 16.x

  Show description for 34678c8
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  34678c8

• python-zeroconf 0.75.0 strict=False for _nmos-registration._tcp

  Show description for db8c8cf
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  db8c8cf

• Python 3.6 (and 3.7!) is End-Of-Life, and so is Node.js 14.x

  Show description for 6cb8f7c
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  6cb8f7c

• Pass IP address of API under test to testssl.sh

  Show description for b3008ad
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  b3008ad

• Add MOCK_SERVICES_WARM_UP_DELAY

  Show description for 353e746
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  353e746

• Delay running tests until after MOCK_SERVICES_WARM_UP_DELAY

  Show description for 23110f1
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  23110f1

• Tweak message

  Show description for ab07908
  

  garethsb

  authored and

  jonathan-r-thorpe

  committed
  ab07908

• Fixed send_command function as per suggestion from @alabou

  Show description for 049d674
  jonathan-r-thorpe

  committed
  049d674

Commits on Oct 24, 2023

• Add config option to bypass IS12 interactive testing

  jonathan-r-thorpe

  committed
  b165316

• Add constraints classes

  jonathan-r-thorpe

  committed
  19f71a8

• added xeger library

  jonathan-r-thorpe

  committed
  b73cd22

• Add IS1202 test suite

  jonathan-r-thorpe

  committed
  ddc17d7

Commits on Oct 31, 2023

• Add primitives to datatype models

  jonathan-r-thorpe

  committed
  b081bb0

Commits on Nov 6, 2023

• Enhance test_35 (subscription and notification mechanism test) to all… (#839)

  Show description for aa6faa9
  

  3 peopleauthored
  aa6faa9

• Tweaked test_35

  jonathan-r-thorpe

  committed
  d862805

Commits on Nov 7, 2023

• Pass **kwargs to ControllerTest

  jonathan-r-thorpe

  committed
  6dd111d

• Pass **kwargs to ControllerTest

  jonathan-r-thorpe

  committed
  19d5b86

Commits on Nov 8, 2023

• Test sequences; remove non UTF-8 character from comment

  jonathan-r-thorpe

  committed
  e581283

• Added constrained sequence tests

  jonathan-r-thorpe

  committed
  ce8afc0

• Additional sequence checks

  jonathan-r-thorpe

  committed
  a0937e1

Commits on Nov 10, 2023

• Add runtime constraints to device model. Fix handling of primitive types in schema generation

  jonathan-r-thorpe

  committed
  7f84e40

Commits on Nov 14, 2023

• Added property and datatype constraints checking

  jonathan-r-thorpe

  committed
  a7af313

• Removed disabled tests

  jonathan-r-thorpe

  committed
  fe032c0

• make datatype_schemas an object property

  jonathan-r-thorpe

  committed
  995e292

• Check for default value sequences when validating class descriptors

  Show description for cedacb8
  jonathan-r-thorpe

  committed
  cedacb8

• Flake8 formatting

  jonathan-r-thorpe

  committed
  1bb41bd

• Cache class_manager. Simplify constraint checking.

  jonathan-r-thorpe

  committed
  8409b8e

• Merge remote-tracking branch 'nmos-testing-amwa/is-12' into is-12-invasive

  jonathan-r-thorpe

  committed
  e0c6d7f

Commits on Nov 15, 2023

• Fix Block object construction. Fix Test_02

  jonathan-r-thorpe

  committed
  30d8059

• add missing primitive descriptor path

  jonathan-r-thorpe

  committed
  72d070b

• reorder specs in NMOS Testing interface

  jonathan-r-thorpe

  committed
  9a0f6bd

• Fix constraint violation error message

  jonathan-r-thorpe

  committed
  c5d43f7

• Disable Testing Facade host and port if not interactively testing

  jonathan-r-thorpe

  committed
  83afc3b

• Flake8 formatting

  jonathan-r-thorpe

  committed
  96aaaa9

• Fix constraint violation error message

  Show description for c8caa23
  jonathan-r-thorpe

  committed
  c8caa23

Commits on Nov 16, 2023

• Refactor property testing

  jonathan-r-thorpe

  committed
  049d62a

• Handle property and method deprecated status codes

  jonathan-r-thorpe

  committed
  5f2cb63

Commits on Nov 17, 2023

• Check deprecated properties

  jonathan-r-thorpe

  committed
  7c93b49

• Report unchecked device model in tests

  jonathan-r-thorpe

  committed
  12f556b

• Renumbered tests

  jonathan-r-thorpe

  committed
  738021f

• updated spec URLs in comments

  jonathan-r-thorpe

  committed
  ef33d0f

• Handle property and method deprecated status codes

  Show description for 23b3512
  jonathan-r-thorpe

  committed
  23b3512

• Check deprecated properties

  Show description for 0841000
  jonathan-r-thorpe

  committed
  0841000

• Report unchecked device model in tests

  Show description for 63bafa7
  jonathan-r-thorpe

  committed
  63bafa7

• Renumbered tests

  Show description for e673449
  jonathan-r-thorpe

  committed
  e673449

• updated spec URLs in comments

  Show description for dc2d30a
  jonathan-r-thorpe

  committed
  dc2d30a

• Merge remote-tracking branch 'origin/is-12-invasive' into is-12

  jonathan-r-thorpe

  committed
  8234809

• Updated to conform merged branch

  jonathan-r-thorpe

  committed
  088b22e

Commits on Dec 18, 2023

• Guard against missing class properties

  jonathan-r-thorpe

  committed
  af8d55e

• Add annotation to check properties test

  jonathan-r-thorpe

  committed
  3640690

• Better error reporting for test_22 and test_23

  jonathan-r-thorpe

  committed
  b53b562

• formatting

  jonathan-r-thorpe

  committed
  d5348e0

Commits on Jan 10, 2024

• Fix spelling mistake in interactive testing config

  jonathan-r-thorpe

  committed
  3fa3fed

Commits on Jan 12, 2024

• Fixed Blocks with no members being set to None

  jonathan-r-thorpe

  committed
  b1c91f4

• Fixed config spelling error

  jonathan-r-thorpe

  committed
  09f2d39

• Rename _execute_command to execute_command

  jonathan-r-thorpe

  committed
  b5205ac

• Test discovered methods in Device Model

  jonathan-r-thorpe

  committed
  35cde03

Commits on Feb 7, 2024

• Only test non-standard methods.

  jonathan-r-thorpe

  committed
  fd8c9d3

Commits on Feb 14, 2024

• Moved some functions to IS12Utils

  jonathan-r-thorpe

  committed
  95fe9e5

Commits on Feb 26, 2024

• Added more error reporting

  jonathan-r-thorpe

  committed
  245e3c6

Commits on Feb 28, 2024

• Guard against ControllerTest being run in non-interactive mode

  jonathan-r-thorpe

  committed
  3483863

Commits on Mar 6, 2024

• Added check constraints hierarchy test

  jonathan-r-thorpe

  committed
  5a15d84

• Fixed do_check_property_test

  jonathan-r-thorpe

  committed
  3260af4

• Add check read only properties and sequences

  jonathan-r-thorpe

  committed
  98b0a64

• Made sequence tests tolerant to empty or null sequences

  jonathan-r-thorpe

  committed
  3fad9a7

• Typo and formatting

  jonathan-r-thorpe

  committed
  a5350cd

Commits on Apr 26, 2024

• Initial add of IS-14-01 test suite

  Show description for 5ba312d
  jonathan-r-thorpe

  committed
  5ba312d

• Flake8 formatting

  jonathan-r-thorpe

  committed
  af0c67a

Commits on Apr 29, 2024

• Added Node API to IS-14 (for testing control endpoint)

  jonathan-r-thorpe

  committed
  7405325

• Initial refactoring

  jonathan-r-thorpe

  committed
  55eeb69

• Added additional specs to IS-14-01 test suite definition

  jonathan-r-thorpe

  committed
  7bb6a5f

• Move common functions from IS12Utils to MS05Utils

  jonathan-r-thorpe

  committed
  a9ed819

• Delegate to MS0501Test for auto tests

  jonathan-r-thorpe

  committed
  03d411a

• API key now required when loading resources

  jonathan-r-thorpe

  committed
  2fb54b2

• Delegate to MS0501Test for auto tests

  jonathan-r-thorpe

  committed
  0062040

• flake8 formatting

  jonathan-r-thorpe

  committed
  84959a6

Commits on May 1, 2024

• Move MS-05 tests into MS0501Test

  Show description for 95a1eac
  jonathan-r-thorpe

  committed
  95a1eac

• Fixed get_sequence_item

  jonathan-r-thorpe

  committed
  ee4888a

Commits on May 2, 2024

• Refactor IS-14 API utility functions

  jonathan-r-thorpe

  committed
  223b29d

• generalize get_control_class and get_datatype. Improve error handling.

  jonathan-r-thorpe

  committed
  2dd9973

• Implement IS-14 http request helper functions

  jonathan-r-thorpe

  committed
  089a2b0

• Formatting

  jonathan-r-thorpe

  committed
  dc9608a

Commits on May 3, 2024

• Moved error handling tests to MS0501Test

  jonathan-r-thorpe

  committed
  80dedb0

• Add invoke method utility function.

  Show description for 36cec82
  jonathan-r-thorpe

  committed
  36cec82

• Do not cache spec_branch in construction

  jonathan-r-thorpe

  committed
  0e2f8f5

• Simplify utilities initialization

  jonathan-r-thorpe

  committed
  ffc7ea0

• Add reset so that state is reset on set_up_tests

  jonathan-r-thorpe

  committed
  1599e8d

• Initiailize IS-12 websocket on set_up_tests so failure is reported on the UI

  jonathan-r-thorpe

  committed
  d5b5851

• make base class generic

  jonathan-r-thorpe

  committed
  ddb1e7f

• Updated comments

  jonathan-r-thorpe

  committed
  ee57347

Commits on May 8, 2024

• Changed " embedd3ed in f-strings to '

  jonathan-r-thorpe

  committed
  9813fba

• Changed SET to PUT

  jonathan-r-thorpe

  committed
  e9aa570

• Changed " embedded in f-strings to '

  jonathan-r-thorpe

  committed
  c3d58d9

Commits on May 9, 2024

• Fix IS-14 utility functions

  jonathan-r-thorpe

  committed
  37301ea

• Add missing sequence methods

  jonathan-r-thorpe

  committed
  bad7453

• Add role_path to get_sequence_length call

  jonathan-r-thorpe

  committed
  f9aa23a

Commits on Jun 3, 2024

• Add device-configuration feature set to feature set tests

  jonathan-r-thorpe

  committed
  5eda47f

• Add additional validation of class descriptors

  jonathan-r-thorpe

  committed
  e228800

Commits on Sep 9, 2024

• Add is_block helper.

  jonathan-r-thorpe

  committed
  80bcca4

• Add initial IS-14 tests

  jonathan-r-thorpe

  committed
  013849d

Commits on Sep 13, 2024

• Improve error reporting

  jonathan-r-thorpe

  committed
  b32a401

Commits on Oct 11, 2024

• Fix linting errors

  jonathan-r-thorpe

  committed
  d96d424

• Add testing of enum ans struct sequences

  Show description for d276ec4
  jonathan-r-thorpe

  committed
  d276ec4

• Moved _get_datatyoe_schema into MS05Utils

  jonathan-r-thorpe

  committed
  91cf7a5

Commits on Oct 14, 2024

• Make TestMetadata an inner class

  jonathan-r-thorpe

  committed
  884d205

• Use TestMetadata inner class. Remove unneeded data members.

  jonathan-r-thorpe

  committed
  d9142a2

Commits on Oct 15, 2024

• Added NcDescriptor and NcPropertyDescriptor classes

  jonathan-r-thorpe

  committed
  8beaa9a

• Refactor to use NcPropertyDescriptor class

  jonathan-r-thorpe

  committed
  3e336bf

• Refactored NcPropertyId and NcTouchpoint types

  jonathan-r-thorpe

  committed
  824678a

Commits on Oct 18, 2024

• Refactor dictionaries into classes

  jonathan-r-thorpe

  committed
  dd33d4c

• Update formatting of strings

  jonathan-r-thorpe

  committed
  a5d9ebc

• Consistent quotes

  jonathan-r-thorpe

  committed
  44b2724

• Renamed test_data directory from IS1201 to MS05

  jonathan-r-thorpe

  committed
  958d962

• Refactored names of functions, improved consistency of strings

  jonathan-r-thorpe

  committed
  9673baa

Commits on Oct 21, 2024

• Create NcFieldDescriptor, NcParameterDescriptor and NcMethodDescriptor classes

  jonathan-r-thorpe

  committed
  798dcad

Commits on Oct 22, 2024

• Introduced constraint domain classes. Refactored test logic.

  jonathan-r-thorpe

  committed
  19008fc

• Handle 'any' datatype in method descriptors

  jonathan-r-thorpe

  committed
  5983e67

Commits on Oct 23, 2024

• Consistent formatting of strings. Refactor to use domain classes

  jonathan-r-thorpe

  committed
  d94de38

• Utility functions to format class ids and role paths. Create NcMethosId class. Replace "context" parameters with role paths

  jonathan-r-thorpe

  committed
  29310f8

Commits on Oct 24, 2024

• Remove context parameter and use role_path instead

  jonathan-r-thorpe

  committed
  f5a5e42

Commits on Nov 1, 2024

• Remove MS-05-02 error handling from IS12Utils

  jonathan-r-thorpe

  committed
  34c312b

• Add NcMethodResult types

  jonathan-r-thorpe

  committed
  b36a657

• Remove redundant imports

  jonathan-r-thorpe

  committed
  ac99d1b

• Formatting

  jonathan-r-thorpe

  committed
  1184d46

• Handle MS-05-02 errors

  jonathan-r-thorpe

  committed
  4acb74e

• get_property returns NcMethodResult object

  jonathan-r-thorpe

  committed
  dad1335

• set_property returns NcMethodResult

  jonathan-r-thorpe

  committed
  12c6f3a

• invoke_method returns NcMethodResult

  jonathan-r-thorpe

  committed
  bd4099c

• get_sequence_item returns NcMethodResult

  jonathan-r-thorpe

  committed
  ebd79bc

Commits on Nov 2, 2024

• get_sequence_length returns NcMethodResult

  jonathan-r-thorpe

  committed
  cc3857a

• set_sequence_item returns NcMethodResult

  jonathan-r-thorpe

  committed
  3b80b6f

• add_sequence_item returns NcMethodResultId

  jonathan-r-thorpe

  committed
  9dcd93d

• remove_sequence_item returns NcMethodResult

  jonathan-r-thorpe

  committed
  9424fc3

Commits on Nov 3, 2024

• get_member_descriptor returns NcMethodResult

  jonathan-r-thorpe

  committed
  e66fa76

• find_members_by_path returns NcMethodResult

  jonathan-r-thorpe

  committed
  f297309

Commits on Nov 4, 2024

• find_members_by_role returns NcMethodResult

  jonathan-r-thorpe

  committed
  8ae7779

• find_members_by_class_id returns NcMethodResult

  jonathan-r-thorpe

  committed
  d061f4f

• get_control_class returns NcMethodResult

  jonathan-r-thorpe

  committed
  8dbff44

• get_datatype returns NcMethodResult

  jonathan-r-thorpe

  committed
  ff03fd3

• Use NcPropertyId and NcMethodId instead of raw dict

  jonathan-r-thorpe

  committed
  33f499c

• Implement IS12Notification class

  jonathan-r-thorpe

  committed
  9455fd1

• Implement NcPropertyChangedEventData

  jonathan-r-thorpe

  committed
  82ebd13

• Implement IS12CommandResponse class

  jonathan-r-thorpe

  committed
  ec3c2a1

• Check subscription response

  jonathan-r-thorpe

  committed
  c5b5d31

Commits on Nov 5, 2024

• Refactor checking of Block members

  jonathan-r-thorpe

  committed
  e397fde

• Formatting

  jonathan-r-thorpe

  committed
  edaa948

• Implement IS12Error class. Formatting

  jonathan-r-thorpe

  committed
  411d9c4

Commits on Nov 6, 2024

• Refine IS14Utils error handling

  jonathan-r-thorpe

  committed
  80ed30f

Commits on Nov 7, 2024

• Fix method invocation JSON

  jonathan-r-thorpe

  committed
  9148e63

• Simplify property constraints checking

  jonathan-r-thorpe

  committed
  df3bf37

• Fix dicts embedded in f-strings

  jonathan-r-thorpe

  committed
  d989b4b

• Add spec links to test failures

  jonathan-r-thorpe

  committed
  0a0ab71

• Fix dicts embedded in f-strings

  jonathan-r-thorpe

  committed
  f74eb31

• Fix embedded code strings in f-strings

  jonathan-r-thorpe

  committed
  2bac92d

• Fix embedded code strings in f-strings

  jonathan-r-thorpe

  committed
  e391efa

• Format error messages

  jonathan-r-thorpe

  committed
  916336a

Commits on Nov 11, 2024

• Refactored constraints testing

  jonathan-r-thorpe

  committed
  f1261d1

• Improve notifications handling

  

  alabou

  authored and

  jonathan-r-thorpe

  committed
  05f24f2

• Flake8 formatting

  jonathan-r-thorpe

  committed
  6777734

Commits on Nov 12, 2024

• Rename context to error_msg_base

  jonathan-r-thorpe

  committed
  9d1d3b6

• consistent format of error messages

  jonathan-r-thorpe

  committed
  ea7a5fa

Commits on Nov 13, 2024

• Better handling of IS-12 errors

  jonathan-r-thorpe

  committed
  592b4f6

• Add robustness for missing classes and datatype definitions

  jonathan-r-thorpe

  committed
  2cbb4c0

Commits on Nov 15, 2024

• Consistent formatting of error messages

  jonathan-r-thorpe

  committed
  dbdc5df

Commits on Nov 19, 2024

• Improve constraints testing

  jonathan-r-thorpe

  committed
  e53fa8e

• Add NcEventDescriptor

  jonathan-r-thorpe

  committed
  5da7575

• Validate parameter constraints. Check for illegal non-standard datatypes

  jonathan-r-thorpe

  committed
  6024577

• Minor bug fixes

  jonathan-r-thorpe

  committed
  0322fb7

• Merge branch 'master' into is-12

  jonathan-r-thorpe

  committed
  2d49bfa

Commits on Nov 20, 2024

• Update comments

  jonathan-r-thorpe

  committed
  a987151

• Clearly indicate private helper functions

  jonathan-r-thorpe

  committed
  becf193

• Simplify constraints checking tests

  jonathan-r-thorpe

  committed
  edb91e4

• Clearly indicate private helper functions

  jonathan-r-thorpe

  committed
  d140e05

• Improve error message

  jonathan-r-thorpe

  committed
  88ebc67

Commits on Nov 21, 2024

• Allow calling test to explicitly set test method name reported in Testing Facade

  jonathan-r-thorpe

  committed
  3d85279

• Fix interaction with Testing Facade

  jonathan-r-thorpe

  committed
  3e6c263

• Format error message

  jonathan-r-thorpe

  committed
  1c6ee65

• Remove IS-14 test suites

  jonathan-r-thorpe

  committed
  089a33e

Commits on Nov 28, 2024

• Improve descriptor validation error reporting

  jonathan-r-thorpe

  committed
  d1403e2