common-workflow-language / common-workflow-language/cwltool

bad user experience if an array is provided to a File's path

Open
#372 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

I am trying to describe a tool that inputs many FASTQ files and outputs only one file. However, the cwltool fails when I define an array of File as input. I wrote a minimal example to reproduce this error using `cat` command.

`cat.cwl` file:

``` yml
cwlVersion: v1.0
class: CommandLineTool
baseCommand: [cat]
stdout: output.txt

inputs:
files:
type: File[]
inputBinding: {}

outputs:
output:
type: stdout
```

`cat-test.yml` file:

``` yml
files:
class: File
path: [file1.txt, file2.txt, file3.txt]
```

Command:

``` bash
cwl-runner cat.cwl cat-test.yml
```

Error:

```
/usr/local/bin/cwl-runner 1.0.20170413194156
Resolved 'cat.cwl' to '/home/welliton/cat.cwl'
Traceback (most recent call last):
File "/usr/local/bin/cwl-runner", line 9, in
load_entry_point('cwl-runner==1.0', 'console_scripts', 'cwl-runner')()
File "/usr/local/lib/python2.7/dist-packages/cwltool/main.py", line 753, in main
fetcher_constructor=fetcher_constructor)
File "/usr/local/lib/python2.7/dist-packages/cwltool/main.py", line 509, in load_job_order
normalizeFilesDirs(job_order_object)
File "/usr/local/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 78, in normalizeFilesDirs
adjustFileObjs(job, addLocation)
File "/usr/local/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 41, in adjustFileObjs
adjustFileObjs(rec[d], op)
File "/usr/local/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 39, in adjustFileObjs
op(rec)
File "/usr/local/lib/python2.7/dist-packages/cwltool/pathmapper.py", line 75, in addLocation
parse = urllib.parse.urlparse(d["location"])
File "/usr/lib/python2.7/urlparse.py", line 143, in urlparse
tuple = urlsplit(url, scheme, allow_fragments)
File "/usr/lib/python2.7/urlparse.py", line 176, in urlsplit
cached = _parse_cache.get(key, None)
TypeError: unhashable type: 'CommentedSeq'
````

Is there any example of using an array of File as input? I followed the [user guide](http://www.commonwl.org/v1.0/UserGuide.html#Array_inputs).

Contributor guide

Open the contributing guide

Research direction

Start with cwltool/pathmapper.py, especially normalizeFilesDirs, adjustFileObjs, and addLocation, then reproduce the failure with the shown cat.cwl, cat-test.yml, and cwl-runner command. Confirm that an array of File paths is accepted without the unhashable CommentedSeq error and that the cat example completes successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.