You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Test instances that are missing mandatory information (broadly defined), like a test case name, or a failure_type (if passed is false) should never make it into the database and should be spat back to the submitter while accepting the rest. Might also spit back unrecognized keys since they are likely wrong, or at least as a reminder to the developer to make them do something on the backend.
Background
mesa handles creating the YAML files, and mesa_test handles submitting them. In principle, mesa_test could check the YAML files for consistency so it never makes it to the server, but that would require mesa_test to be "smart", and more importantly, updated. Kicking this up to the testhub means that mesa_test can remain a blameless messenger between the testing scripts of mesa and the database in the testhub.
The text was updated successfully, but these errors were encountered:
Test instances that are missing mandatory information (broadly defined), like a test case name, or a
failure_type
(ifpassed
isfalse
) should never make it into the database and should be spat back to the submitter while accepting the rest. Might also spit back unrecognized keys since they are likely wrong, or at least as a reminder to the developer to make them do something on the backend.Background
mesa
handles creating the YAML files, andmesa_test
handles submitting them. In principle,mesa_test
could check the YAML files for consistency so it never makes it to the server, but that would requiremesa_test
to be "smart", and more importantly, updated. Kicking this up to the testhub means thatmesa_test
can remain a blameless messenger between the testing scripts ofmesa
and the database in the testhub.The text was updated successfully, but these errors were encountered: