common-workflow-language / common-workflow-language/cwltool
WorkflowStep.{requirements,hints} are unable to see inputs that aren't apart of the WorkflowStep.run procces
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
## Expected Behavior
Like for `WorkflowStep.in[].valueFrom`, CWL parameter references and CWL expressions at the level of a WorkflowStep should have visibility to the `WorkflowStep` inputs.
## Actual Behavior
`cwltool` mixes `WorkflowStep.{requirements, hints}` with the underlying Process where they are evaluated later whenthe `WorkflowStep.in` inputs are not consulted.
## Workflow Code
> wf_step_extra_inputs.cwl
``` cwl
#!/usr/bin/cwl-runner
cwlVersion: v1.0
class: Workflow
inputs:
greeting:
type: string
default: hello!
requirements:
StepInputExpressionRequirement: {}
steps:
greet:
in:
my_greeting: greeting
my_other_greeting:
source: greeting
valueFrom: "Yo, $(inputs.my_greeting)"
your_greeting:
valueFrom: Salutations!
out: [ result ]
requirements:
EnvVarRequirement:
envDef:
GREETING1: $(inputs.my_greeting)
GREETING2: $(inputs.my_other_greeting)
GREETING3: $(inputs.your_greeting)
run:
class: CommandLineTool
requirements:
ShellCommandRequirement: {}
inputs:
your_greeting: string
baseCommand: bash
arguments:
- -c
- echo \${GREETING1:-fail1} \${GREETING2:-fail2} \${GREETING3:-fail3}
outputs:
result: stdout
outputs:
result:
type: File
outputSource: greet/result
```
## Full Traceback
```pytb
$ cwltool --debug --disable-color wf_step_extra_inputs.cwl
INFO /home/michael/cwltool/env3.7/bin/cwltool 1.0.20190621234233
INFO Resolved 'wf_step_extra_inputs.cwl' to 'file:///home/michael/cwltool/wf_step_extra_inputs.cwl'
WARNING Workflow checker warning:
wf_step_extra_inputs.cwl:13:7: 'my_greeting' is not an input parameter of ordereddict([('class',
'CommandLineTool'), ('requirements', [ordereddict([('class',
'ShellCommandRequirement')])]), ('inputs', [ordereddict([('type',
'string'), ('id',
'file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/ff78fb6c-f363-40b9-b37b-b9d55836a548/your_greeting')])]),
('baseCommand', 'bash'), ('arguments', ['-c', 'echo
\\${GREETING1:-fail1} \\${GREETING2:-fail2}
\\${GREETING3:-fail3}']), ('outputs', [ordereddict([('type',
'File'), ('id',
'file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/ff78fb6c-f363-40b9-b37b-b9d55836a548/result'),
('outputBinding', ordereddict([('glob',
'2110fa1703fd10042bcde9f4b8ff61fbe57530e1')]))])]), ('id',
'file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/ff78fb6c-f363-40b9-b37b-b9d55836a548'),
('stdout', '2110fa1703fd10042bcde9f4b8ff61fbe57530e1')]), expected
your_greeting
DEBUG Parsed job order from command line: {
"id": "wf_step_extra_inputs.cwl",
"greeting": "hello!"
}
DEBUG [workflow ] initialized from file:///home/michael/cwltool/wf_step_extra_inputs.cwl
INFO [workflow ] start
DEBUG [workflow ] {
"greeting": "hello!"
}
INFO [workflow ] starting step greet
DEBUG [job step greet] job input {
"file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/my_greeting": "hello!",
"file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/my_other_greeting": "hello!",
"file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/your_greeting": null
}
DEBUG [job step greet] evaluated job input to {
"file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/my_greeting": "hello!",
"file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/my_other_greeting": "Yo, hello!",
"file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/your_greeting": "Salutations!"
}
INFO [step greet] start
DEBUG [job greet] initializing from file:///home/michael/cwltool/wf_step_extra_inputs.cwl#greet/ff78fb6c-f363-40b9-b37b-b9d55836a548 as part of step greet
DEBUG [job greet] {
"your_greeting": "Salutations!"
}
DEBUG [job greet] path mappings is {}
DEBUG [job greet] command line bindings is [
{
"position": [
-1000000,
0
],
"datum": "bash"
},
{
"position": [
0,
0
],
"datum": "-c"
},
{
"position": [
0,
1
],
"valueFrom": "echo \\${GREETING1:-fail1} \\${GREETING2:-fail2} \\${GREETING3:-fail3}"
}
]
ERROR Exception on step 'greet'
ERROR [step greet] Cannot make job: Expression evaluation error:
Syntax error in parameter reference 'inputs.my_greeting': inputs does not contain key 'my_greeting'. This could be due to using Javascript code without specifying InlineJavascriptRequirement.
DEBUG
Traceback (most recent call last):
File "/home/michael/cwltool/cwltool/expression.py", line 296, in do_eval
strip_whitespace=strip_whitespace)
File "/home/michael/cwltool/cwltool/expression.py", line 230, in interpolate
debug=debug, js_console=js_console)
File "/home/michael/cwltool/cwltool/expression.py", line 202, in evaluator
(ex[1:-1], expression_parse_exception))
cwltool.sandboxjs.JavascriptException: Syntax error in parameter reference 'inputs.my_greeting': inputs does not contain key 'my_greeting'. This could be due to using Javascript code without specifying InlineJavascriptRequirement.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/michael/cwltool/cwltool/workflow.py", line 474, in job
for newjob in step.iterable:
File "/home/michael/cwltool/cwltool/workflow.py", line 408, in try_make_job
for j in jobs:
File "/home/michael/cwltool/cwltool/workflow.py", line 207, in job
for j in self.step.job(joborder, output_callback, runtimeContext):
File "/home/michael/cwltool/cwltool/workflow.py", line 801, in job
runtimeContext):
File "/home/michael/cwltool/cwltool/command_line_tool.py", line 600, in job
j.environment[t["envName"]] = builder.do_eval(t["envValue"])
File "/home/michael/cwltool/cwltool/builder.py", line 452, in do_eval
strip_whitespace=strip_whitespace)
File "/home/michael/cwltool/cwltool/expression.py", line 299, in do_eval
raise_from(WorkflowException("Expression evaluation error:\n%s" % Text(e)), e)
File "/home/michael/cwltool/env3.7/lib/python3.7/site-packages/future/utils/__init__.py", line 400, in raise_from
exec(execstr, myglobals, mylocals)
File "", line 1, in
cwltool.errors.WorkflowException: Expression evaluation error:
Syntax error in parameter reference 'inputs.my_greeting': inputs does not contain key 'my_greeting'. This could be due to using Javascript code without specifying InlineJavascriptRequirement.
INFO [workflow ] completed permanentFail
DEBUG [workflow ] {
"result": null
}
{
"result": null
}
WARNING Final process status is permanentFail
```
## Your Environment
* cwltool 1.0.20190621234233
Contributor guide
Assessment
This issue has not been assessed yet.