common-workflow-language / common-workflow-language/cwltool

CWLtool issue with passing a list of scatter/gathered outputs into a program that accepts a list

Đang mở
#17 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
376
Fork
255
Merge trung bình
2 ngày 7 giờ
Pull request đã merge (30 ngày)
12

Mô tả

This works:

```
#!/usr/bin/env cwl-runner
class: Workflow
requirements:
- class: ScatterFeatureRequirement
inputs:
- id: "#mutation"
type: string
- id: "#normalin"
type: File
- id: "#tumorin"
type: File
outputs:
- id: "#outfile"
type: File
source: "#wc.outfile"
steps:
- id: "#greptumor"
run: {import: grep.cwl.yaml}
#scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#tumorin"
outputs:
- id: "#greptumor.outfile"
- id: "#grepnormal"
run: {import: grep.cwl.yaml}
#scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#normalin"
outputs:
- id: "#grepnormal.outfile"
- id: "#wc"
run: {import: wc.cwl.yaml}
inputs:
- id: "#wc.infile"
source: ["#grepnormal.outfile", "#greptumor.outfile"]
outputs:
- id: "#wc.outfile"
```

This does not work, when I try to have it work on multiple normal/tumor input files into grep scattered:

```
#!/usr/bin/env cwl-runner
class: Workflow
requirements:
- class: ScatterFeatureRequirement
inputs:
- id: "#mutation"
type: string
- id: "#normalin"
type: {type: array, items: File}
- id: "#tumorin"
type: {type: array, items: File}
outputs:
- id: "#outfile"
type: File
source: "#wc.outfile"
steps:
- id: "#greptumor"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#tumorin"
outputs:
- id: "#greptumor.outfile"
- id: "#grepnormal"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#normalin"
outputs:
- id: "#grepnormal.outfile"
- id: "#wc"
run: {import: wc.cwl.yaml}
inputs:
- id: "#wc.infile"
source: ["#grepnormal.outfile", "#greptumor.outfile"]
outputs:
- id: "#wc.outfile"
```

Interestingly though, this small change does work, when you only pass one of the scatter/gathered inputs into the final wc.

```
#!/usr/bin/env cwl-runner
class: Workflow
requirements:
- class: ScatterFeatureRequirement
inputs:
- id: "#mutation"
type: string
- id: "#normalin"
type: {type: array, items: File}
- id: "#tumorin"
type: {type: array, items: File}
outputs:
- id: "#outfile"
type: File
source: "#wc.outfile"
steps:
- id: "#greptumor"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#tumorin"
outputs:
- id: "#greptumor.outfile"
- id: "#grepnormal"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#normalin"
outputs:
- id: "#grepnormal.outfile"
- id: "#wc"
run: {import: wc.cwl.yaml}
inputs:
- id: "#wc.infile"
source: ["#grepnormal.outfile"]
outputs:
- id: "#wc.outfile"
```

So it seems to me that this is an issue only when you use the program accepting an array of inputs and you want to pass two of those scatter/gathered things into a final output. Maybe the data structure being output in the scatter/gather is weird when you put it into a list? The error is this about a dict being expected:

```
> cwltool mutationfinder3.cwl.yaml --mutation GCATCCA --normalin normal.fastq --tumorin tumor.fastq
/anaconda/bin/cwltool 1.0.20151026181844
[job 4386765520] /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpm8fJ5E$ grep GCATCCA /Users/john/workspaces/commonwl_examples/t790m_detector/normal.fastq > /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpm8fJ5E/out.txt
[job 4386765840] /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpUclOsb$ grep GCATCCA /Users/john/workspaces/commonwl_examples/t790m_detector/tumor.fastq > /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpUclOsb/out.txt
Workflow error:
Error validating input record, could not validate field `infile` because
At position 0
`[{'checksum': 'sha1$e62341a47da9cb155ec0f531b525a97d2bd86455',
'class': 'File',
'path': '/var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpm8fJ5E/out.txt'[...]`
is not a dict
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.