common-workflow-language / common-workflow-language/cwltool

Secondary Files required

Open
#1,151 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

## Expected Behavior
cwl v1.0 file with json that was able to run before with cwltool 1.0.20181217162649 should be able to be run with cwltool 1.0.20190621234233 . The secondary files are missing but still ran sucessfully before

## Actual Behavior
```
ERROR Workflow error, try again with --debug for more information:
Missing required secondary file 'gdc-viral.dict.fai' from file object: {
"basename": "gdc-viral.dict",
"nameroot": "gdc-viral",
"nameext": ".dict",
"location": "file:///home/gluu/dockstore/dockstore-client/src/test/resources/biodata/viral/gdc-viral.dict",
"secondaryFiles": [],
"class": "File",
"size": 0
}
```

## Workflow Code
Workflow: https://github.com/dockstore/dockstore/blob/develop/dockstore-client/src/test/resources/directory/1st-workflow-no-secondary-in-workflow.cwl
JSON: https://github.com/dockstore/dockstore/blob/develop/dockstore-client/src/test/resources/directory/1st-workflow-job.json

## Full Traceback
```pytb
INFO /home/gluu/dockstore/venv/bin/cwltool 1.0.20190621234233
INFO Resolved '/home/gluu/dockstore/dockstore-client/src/test/resources/directory/1st-workflow-no-secondary-in-workflow.cwl' to 'file:///home/gluu/dockstore/dockstore-client/src/test/resources/directory/1st-workflow-no-secondary-in-workflow.cwl'
ERROR Workflow error:
Missing required secondary file 'gdc-viral.dict.fai' from file object: {
"basename": "gdc-viral.dict",
"nameroot": "gdc-viral",
"nameext": ".dict",
"location": "file:///home/gluu/dockstore/dockstore-client/src/test/resources/biodata/viral/gdc-viral.dict",
"secondaryFiles": [],
"class": "File",
"size": 0
}
Traceback (most recent call last):
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/main.py", line 823, in main
logger=_logger)
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/executors.py", line 44, in __call__
return self.execute(*args, **kwargs)
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/executors.py", line 103, in execute
self.run_jobs(process, job_order_object, logger, runtime_context)
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/executors.py", line 165, in run_jobs
for job in jobiter:
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/workflow.py", line 578, in job
builder = self._init_job(job_order, runtimeContext)
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/process.py", line 719, in _init_job
discover_secondaryFiles=getdefault(runtime_context.toplevel, False)))
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/builder.py", line 277, in bind_input
bindings.extend(self.bind_input(f, datum[f["name"]], lead_pos=lead_pos, tail_pos=f["name"], discover_secondaryFiles=discover_secondaryFiles))
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/builder.py", line 269, in bind_input
bindings.extend(self.bind_input(st, datum, lead_pos=lead_pos, tail_pos=tail_pos, discover_secondaryFiles=discover_secondaryFiles))
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/builder.py", line 295, in bind_input
self.bind_input(itemschema, item, lead_pos=n, tail_pos=tail_pos, discover_secondaryFiles=discover_secondaryFiles))
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/builder.py", line 269, in bind_input
bindings.extend(self.bind_input(st, datum, lead_pos=lead_pos, tail_pos=tail_pos, discover_secondaryFiles=discover_secondaryFiles))
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/builder.py", line 295, in bind_input
self.bind_input(itemschema, item, lead_pos=n, tail_pos=tail_pos, discover_secondaryFiles=discover_secondaryFiles))
File "/home/gluu/dockstore/venv/local/lib/python2.7/site-packages/cwltool/builder.py", line 343, in bind_input
sfname, json_dumps(datum, indent=4)))
WorkflowException: Missing required secondary file 'gdc-viral.dict.fai' from file object: {
"basename": "gdc-viral.dict",
"nameroot": "gdc-viral",
"nameext": ".dict",
"location": "file:///home/gluu/dockstore/dockstore-client/src/test/resources/biodata/viral/gdc-viral.dict",
"secondaryFiles": [],
"class": "File",
"size": 0
}

```

## Your Environment
* cwltool version:
Before: cwltool 1.0.20181217162649
After: cwltool 1.0.20190621234233

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.