ExpressionTool doesn't check output types

[GlassOfWhiskey]

CWL Specification explicitly states that

The Any type validates for any non-null value.

However, cwtool does not fail when a step produces a null output for a port of type Any. Instead, the behavior is correct for input ports. As a concrete example, the following ExpressionTool should clearly fail, but it doesn't.

#!/usr/bin/env cwl-runner

class: ExpressionTool
requirements: { InlineJavascriptRequirement: {} }
cwlVersion: v1.0

inputs: []

outputs:
  output: Any

expression: "$({'output': null })"

Note also that this file is involved in 2 CWL Conformance tests cases, step_input_default_value_overriden_2nd_step_null and step_input_default_value_overriden_2nd_step_null_noexp. Therefore, it is quite crucial to establish the correct behavior here.

[GlassOfWhiskey]

The problem is caused by the fact that cwltool does not apaprently run output tyoe validation after ExpressionTool executions. Conversely, it behaves correctly with CommandLineTool objects.

[GlassOfWhiskey]

Also this ExpressionTool object is not failing, while it should

cwlVersion: v1.2
class: ExpressionTool
requirements:
  InlineJavascriptRequirement: {}
inputs: []
outputs:
  wrong_type_output: int
expression: "$({'wrong_type_output': 'This is a terrible error!!!'})"

This confirms that no output check is currently running for ExpressionTool objects.

Proposed solution:

• Explicitly document the behaviour in CWL v1.2.1 (Document that ExpressionTool outputs are always valid cwl-v1.2#230)
• Correct it in CWL v1.3 and subsequent versions (Test for ExpressionTool outputs validation cwl-v1.3#9)
• Fix the behaviour in cwltool
• Make the upgrader implicitly set Any? type for all ExpressionTool outputs for CWL <= v1.2