common-workflow-language / common-workflow-language/cwltool

GA4GH TRS: linked files fail to resolve

Open
#1,022 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

## Expected Behavior
Template generation or execution should be successful

## Actual Behavior

```
$ cwltool --make-template quay.io/wtsicgp/dockstore-cgpmap:3.0.0
/home/kr2/.local/bin/cwltool 1.0.20181217162649
Resolved 'quay.io/wtsicgp/dockstore-cgpmap:3.0.0' to 'https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor'
Tool definition failed initialization:
https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor:21:5: ('https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml', HTTPError('404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml',))
```

## Workflow Code
Commands instead:

Fails:
```
cwltool --make-template quay.io/wtsicgp/dockstore-cgpmap:3.0.0
```

Success with a full URL (no GA4GH TRS):

```
$ cwltool --make-template https://dockstore.org:443/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor/%2FDockstore.cwl
/home/kr2/.local/bin/cwltool 1.0.20181217162649
sample: a_string # type "string"
reference: # type "File"
class: File
path: a/file/path
mmqcfrac: 0.05 # default value of type "float". (optional)
mmqc: false # type "boolean"
groupinfo: # type "File" (optional)
class: File
path: a/file/path
bwa_idx: # type "File"
class: File
path: a/file/path
bwa: ' -Y -K 100000000' # default value of type "string". (optional)
bams_in: # array of type "File" (optional)
- class: File
path: a/file/path
```

## Full Traceback
```pytb
Traceback (most recent call last):
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 143, in fetch_text
resp.raise_for_status()
File "/home/kr2/.local/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 590, in resolve_ref
doc = self.fetch(doc_url, inject_ids=(not mixin))
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 955, in fetch
text = self.fetch_text(url)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 145, in fetch_text
raise RuntimeError(url, e)
RuntimeError: ('https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml', HTTPError('404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/kr2/.local/lib/python3.6/site-packages/cwltool/main.py", line 661, in main
do_validate=loadingContext.do_validate)
File "/home/kr2/.local/lib/python3.6/site-packages/cwltool/load_tool.py", line 284, in validate_document
workflowobj, fileuri, checklinks=do_validate)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 896, in resolve_all
val, base_url+subscope, file_base=file_base, checklinks=False)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 909, in resolve_all
val, base_url=file_base, checklinks=False)
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/ref_resolver.py", line 590, in resolve_ref
doc = self.fetch(doc_url, inject_ids=(not mixin))
File "/home/kr2/.local/lib/python3.6/site-packages/schema_salad/sourceline.py", line 168, in __exit__
raise self.makeError(six.text_type(exc_value))
RuntimeError: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/descriptor:21:5: ('https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml', HTTPError('404 Client Error: Not Found for url: https://dockstore.org/api/api/ga4gh/v2/tools/quay.io%2Fwtsicgp%2Fdockstore-cgpmap/versions/3.0.0/plain-CWL/cwls/mixins/requirements.yml',))
```

## Your Environment

```
$ cwltool --version
/home/kr2/.local/bin/cwltool 1.0.20181217162649
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.