common-workflow-language / common-workflow-language/cwltool
GA4GH TRS: linked files fail to resolve
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
## Expected Behavior
Template generation or execution should be successful
## Actual Behavior
```
$ cwltool --make-template quay.io/wtsicgp/dockstore-cgpmap:3.0.0
/home/kr2/.local/bin/cwltool 1.0.20181217162649
Resolved 'quay.io/wtsicgp/dockstore-cgpmap:3.0.0' to 'https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor'
Tool definition failed initialization:
https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor:21:5: ('https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml', HTTPError('404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml',))
```
## Workflow Code
Commands instead:
Fails:
```
cwltool --make-template quay.io/wtsicgp/dockstore-cgpmap:3.0.0
```
Success with a full URL (no GA4GH TRS):
```
$ cwltool --make-template https://dockstore.org:443/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor/%2FDockstore.cwl
/home/kr2/.local/bin/cwltool 1.0.20181217162649
sample: a_string # type "string"
reference: # type "File"
class: File
path: a/file/path
mmqcfrac: 0.05 # default value of type "float". (optional)
mmqc: false # type "boolean"
groupinfo: # type "File" (optional)
class: File
path: a/file/path
bwa_idx: # type "File"
class: File
path: a/file/path
bwa: ' -Y -K 100000000' # default value of type "string". (optional)
bams_in: # array of type "File" (optional)
- class: File
path: a/file/path
```
## Full Traceback
```pytb
Traceback (most recent call last):
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 143, in fetch_text
resp.raise_for_status()
File "/home/kr2/.local/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 590, in resolve_ref
doc = self.fetch(doc_url, inject_ids=(not mixin))
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 955, in fetch
text = self.fetch_text(url)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 145, in fetch_text
raise RuntimeError(url, e)
RuntimeError: ('https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml', HTTPError('404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/kr2/.local/lib/python3.6/site-packages/cwltool/main.py", line 661, in main
do_validate=loadingContext.do_validate)
File "/home/kr2/.local/lib/python3.6/site-packages/cwltool/load_tool.py", line 284, in validate_document
workflowobj, fileuri, checklinks=do_validate)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 896, in resolve_all
val, base_url+subscope, file_base=file_base, checklinks=False)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 909, in resolve_all
val, base_url=file_base, checklinks=False)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 590, in resolve_ref
doc = self.fetch(doc_url, inject_ids=(not mixin))
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/sourceline.py", line 168, in __exit__
raise self.makeError(six.text_type(exc_value))
RuntimeError: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor:21:5: ('https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml', HTTPError('404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml',))
```
## Your Environment
```
$ cwltool --version
/home/kr2/.local/bin/cwltool 1.0.20181217162649
```
Contributor guide
Assessment
This issue has not been assessed yet.