common-workflow-language / common-workflow-language/user_guide

Handling of exclusive/dependent inputs

Open
#469 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Common Workflow Language
Stars
42
Forks
65
PR merge metrics
No merged PRs in 30d

Description

More guidance on how to use dependent and mutually exclusive inputs in a multi-stepped workflow would be nice.
I could only find something in the FAQ: [https://www.commonwl.org/user_guide/faq.html#contents](https://www.commonwl.org/user_guide/faq.html#contents)

Some examples for dependent workflows. _Credit goes to @alexiswl_

**command line too cwl:**
```
cwlVersion: v1.2
class: CommandLineTool
inputs:
dependent_parameters:
type:
type: record
name: dependent_parameters
fields:
itemA:
type: string
inputBinding:
prefix: -A
itemB:
type: string
inputBinding:
prefix: -B
outputs:
example_out:
type: stdout
stdout: output.txt
baseCommand: echo
```
If the workflow input type is the same record schema as the tool, you can simply parse them like you do with any object such as a file or directory.
**record_wf.cwl**
```
cwlVersion: v1.2
class: Workflow

inputs:
dependent_parameters:
type:
type: record
name: dependent_parameters
fields:
itemA:
type: string
itemB:
type: string

steps:
record:
run: record.cwl
in:
dependent_parameters: dependent_parameters
out: [example_out]

outputs:
example_out:
type: File
outputSource: record/example_out
```

Things get a little tricky when you need to convert a record type from the directory to the workflow
i.e let's say your workflow has itemC instead of itemB, but your tool has the existing itemA / itemB record type.

One way to convert is by using the valueFrom attribute in the step input (note the requirements update needed to the workflow)
**record_wf.cwl**
```
#!/usr/bin/env cwl-runner
cwlVersion: v1.2
class: Workflow

# Requirements needed for workflow
# valueFrom requires StepInputExpressionRequirement
# While any javascript code present will require the InlineJavascriptRequirement
requirements:
StepInputExpressionRequirement: {}
InlineJavascriptRequirement: {}

inputs:
dependent_parameters:
type:
type: record
name: dependent_parameters
fields:
itemA:
type: string
itemC: # New record type
type: string

steps:
record:
run: record.cwl # Expects old record type
in:
dependent_parameters:
source: dependent_parameters
valueFrom: |
${
return {
"itemA": self.itemA,
"itemB": self.itemC
}
}
out: [example_out]

outputs:
example_out:
type: File
outputSource: record/example_out
```
Let's test this with our input yaml
```
dependent_parameters:
itemA: a_string # type "string
itemC: c_string # type "string"
```
```
cwltool record-wf.cwl input.yaml
```
Gives
```
INFO /usr/bin/cwltool 3.1.20220224085855
INFO Resolved 'record-wf.cwl' to 'file:///tmp/tmp.gkj56GiQ0w/record-wf.cwl'
INFO [workflow ] start
INFO [workflow ] starting step record
INFO [step record] start
INFO [job record] /tmp/oq_5u9f8$ echo \
-A \
a_string \
-B \
c_string > /tmp/oq_5u9f8/output.txt
INFO [job record] completed success
INFO [step record] completed success
INFO [workflow ] completed success
{
"example_out": {
"location": "file:///tmp/tmp.gkj56GiQ0w/output.txt",
"basename": "output.txt",
"class": "File",
"checksum": "sha1$f215e8565d4711ed1fac71938dee09ef7781870f",
"size": 24,
"path": "/tmp/tmp.gkj56GiQ0w/output.txt"
}
}
INFO Final process status is success
```
**output.txt**
```
-A a_string -B c_string
```
Let's instead assume an eggnog-mapper tool requires the inputs input_fasta, data_dir, db and diamond_db, we can instead use the JavaScript expression / valueFrom syntax shown above to collect each of these attributes from the eggnog input record.

For example
```
in:
input_fasta: bakta/sequences_cds
data_dir:
source: eggnog
valueFrom: |
${
return self.data_dir;
}
db:
source: eggnog
valueFrom: |
${
return self.db;
}
diamond_db:
source: eggnog
valueFrom: |
${
return self.diamond_db;
}
```

This last example could be more generalized...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.