common-workflow-language / common-workflow-language/cwltool

CWLtool issue with passing a list of scatter/gathered outputs into a program that accepts a list

Abierto
#17 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
376
Forks
255
Merge medio
2 d 7 h
PR fusionados (30 d)
12

Descripción

This works:

```
#!/usr/bin/env cwl-runner
class: Workflow
requirements:
- class: ScatterFeatureRequirement
inputs:
- id: "#mutation"
type: string
- id: "#normalin"
type: File
- id: "#tumorin"
type: File
outputs:
- id: "#outfile"
type: File
source: "#wc.outfile"
steps:
- id: "#greptumor"
run: {import: grep.cwl.yaml}
#scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#tumorin"
outputs:
- id: "#greptumor.outfile"
- id: "#grepnormal"
run: {import: grep.cwl.yaml}
#scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#normalin"
outputs:
- id: "#grepnormal.outfile"
- id: "#wc"
run: {import: wc.cwl.yaml}
inputs:
- id: "#wc.infile"
source: ["#grepnormal.outfile", "#greptumor.outfile"]
outputs:
- id: "#wc.outfile"
```

This does not work, when I try to have it work on multiple normal/tumor input files into grep scattered:

```
#!/usr/bin/env cwl-runner
class: Workflow
requirements:
- class: ScatterFeatureRequirement
inputs:
- id: "#mutation"
type: string
- id: "#normalin"
type: {type: array, items: File}
- id: "#tumorin"
type: {type: array, items: File}
outputs:
- id: "#outfile"
type: File
source: "#wc.outfile"
steps:
- id: "#greptumor"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#tumorin"
outputs:
- id: "#greptumor.outfile"
- id: "#grepnormal"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#normalin"
outputs:
- id: "#grepnormal.outfile"
- id: "#wc"
run: {import: wc.cwl.yaml}
inputs:
- id: "#wc.infile"
source: ["#grepnormal.outfile", "#greptumor.outfile"]
outputs:
- id: "#wc.outfile"
```

Interestingly though, this small change does work, when you only pass one of the scatter/gathered inputs into the final wc.

```
#!/usr/bin/env cwl-runner
class: Workflow
requirements:
- class: ScatterFeatureRequirement
inputs:
- id: "#mutation"
type: string
- id: "#normalin"
type: {type: array, items: File}
- id: "#tumorin"
type: {type: array, items: File}
outputs:
- id: "#outfile"
type: File
source: "#wc.outfile"
steps:
- id: "#greptumor"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#tumorin"
outputs:
- id: "#greptumor.outfile"
- id: "#grepnormal"
run: {import: grep.cwl.yaml}
scatter: "#grep.infile"
inputs:
- id: "#grep.pattern"
source: "#mutation"
- id: "#grep.infile"
source: "#normalin"
outputs:
- id: "#grepnormal.outfile"
- id: "#wc"
run: {import: wc.cwl.yaml}
inputs:
- id: "#wc.infile"
source: ["#grepnormal.outfile"]
outputs:
- id: "#wc.outfile"
```

So it seems to me that this is an issue only when you use the program accepting an array of inputs and you want to pass two of those scatter/gathered things into a final output. Maybe the data structure being output in the scatter/gather is weird when you put it into a list? The error is this about a dict being expected:

```
> cwltool mutationfinder3.cwl.yaml --mutation GCATCCA --normalin normal.fastq --tumorin tumor.fastq
/anaconda/bin/cwltool 1.0.20151026181844
[job 4386765520] /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpm8fJ5E$ grep GCATCCA /Users/john/workspaces/commonwl_examples/t790m_detector/normal.fastq > /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpm8fJ5E/out.txt
[job 4386765840] /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpUclOsb$ grep GCATCCA /Users/john/workspaces/commonwl_examples/t790m_detector/tumor.fastq > /var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpUclOsb/out.txt
Workflow error:
Error validating input record, could not validate field `infile` because
At position 0
`[{'checksum': 'sha1$e62341a47da9cb155ec0f531b525a97d2bd86455',
'class': 'File',
'path': '/var/folders/8r/g00jq11j2yb586cz04tfpydc0000gn/T/tmpm8fJ5E/out.txt'[...]`
is not a dict
```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.