common-workflow-language / common-workflow-language/cwltool
using scattered value more than once in a step fails
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
## Expected Behavior
the files a, a.extra, b and b.extra should be created
## Actual Behavior
The workflow fails because the entire array is passed to the second input parameter rather than the current element (which in turns causes the workflow to fail because the invalid character).
## Workflow Code
```
#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.0
requirements:
- class: InlineJavascriptRequirement
- class: ScatterFeatureRequirement
- class: StepInputExpressionRequirement
- class: SubworkflowFeatureRequirement
inputs:
parms: string[]
outputs:
crap1:
type: File[]
outputSource: step1/out1
crap2:
type: File[]
outputSource: step1/out2
steps:
step1:
run:
class: CommandLineTool
inputs:
in1:
type: string
inputBinding:
position: 1
in2:
type: string
inputBinding:
position: 1
outputs:
out1:
type: File
outputBinding:
glob: $(inputs.in1)
out2:
type: File
outputBinding:
glob: $(inputs.in2)
baseCommand: touch
scatter: [in1]
in:
in1: parms
in2:
source: parms
valueFrom: >
${
return self + '.extra';
}
out: [out1, out2]
```
## Full Traceback
```
$ cwltool --debug ../src/x.cwl --parms a --parms b
/stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/test/p/bin/cwltool 1.0.20170828135420
Resolved '../src/x.cwl' to 'file:///stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/src/x.cwl'
Parsed job order from command line: {
"parms": [
"a",
"b"
],
"id": "../src/x.cwl"
}
[workflow x.cwl] initialized from file:///stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/src/x.cwl
[workflow x.cwl] start
[workflow x.cwl] starting step step1
[step step1] start
[job step1] initializing from file:///stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/src/x.cwl#step1/0c9519d3-a002-43df-a528-d5f76fb09719 as part of step step1
[job step1] {
"in2": "a,b.extra",
"in1": "a"
}
[job step1] path mappings is {}
[job step1] command line bindings is [
{
"position": [
-1000000,
0
],
"datum": "touch"
},
{
"position": [
1,
"in1"
],
"datum": "a"
},
{
"position": [
1,
"in2"
],
"datum": "a,b.extra"
}
]
[job step1] /stornext/HPCScratch/home/thomas.e/tmp/tmpbH_bov$ touch \
a \
a,b.extra
Error collecting output for parameter 'out2'
Traceback (most recent call last):
File "/stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/test/p/lib/python2.7/site-packages/cwltool/draft2tool.py", line 522, in collect_output_ports
compute_checksum=compute_checksum)
File "/stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/test/p/lib/python2.7/site-packages/cwltool/draft2tool.py", line 600, in collect_output
with fs_access.open(rfile["location"], "rb") as f:
File "/stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/test/p/lib/python2.7/site-packages/cwltool/stdfsaccess.py", line 46, in open
return open(self._abs(fn), mode)
IOError: [Errno 2] No such file or directory: '/stornext/HPCScratch/home/thomas.e/tmp/tmpbH_bov/a%2Cb.extra'
[job step1] Job error:
../src/x.cwl:42:9: Error collecting output for parameter 'out2':
../src/x.cwl:42:9: [Errno 2] No such file or directory: '/stornext/HPCScratch/home/thomas.e/tmp/tmpbH_bov/a%2Cb.extra'
[job step1] completed permanentFail
[job step1] {}
[job step1] Removing input staging directory /stornext/HPCScratch/home/thomas.e/tmp/tmpdLe4Xl
[job step1] Removing temporary directory /stornext/HPCScratch/home/thomas.e/tmp/tmprqKACk
Workflow cannot make any more progress.
Removing intermediate output directory /home/thomas.e/tmp/tmp8e4oYu
Final process status is permanentFail
```
## Your Environment
* cwltool version:
```
$ cwltool --version
/stornext/Home/data/allstaff/t/thomas.e/dev/resistant-falciparum/test/p/bin/cwltool 1.0.20170828135420
```
Contributor guide
Assessment
This issue has not been assessed yet.