common-workflow-language / common-workflow-language/cwltool
Problem with when statements in the workflow: nonetype input for scatter
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
I have 2 following steps in CWL workflow that are executed conditionally (using when statement) and output from one of them is input for the second one, and second one contains scatter. There is a fragment of code here:
```
steps:
list_fastqs_from_dirs_with_ngs_data:
run: ../shared_tools/collections2files/collections2files.cwl
when: $(inputs.run_step === true)
in:
dirs: dirs_with_ngs_data
run_step: run_sort_fastq
out: [files]
sort_fastq:
run: ./sort_fastq/sort_fastq.cwl
when: $(inputs.run_sort_fastq === true)
in:
fastq_file: list_fastqs_from_dirs_with_ngs_data/files
run_sort_fastq: run_sort_fastq
scatter: fastq_file
out: [sorted_fastq]
```
When I run the pipeline and `run_sort_fastq === true` pipeline is completed.
But when `run_sort_fastq === false` pipeline fails, because scatter is empty:
```
ERROR Unhandled exception
Traceback (most recent call last):
File "/usr/local/Caskroom/miniconda/base/lib/python3.9/site-packages/cwltool/workflow_job.py", line 730, in try_make_job
emptyscatter = [
File "/usr/local/Caskroom/miniconda/base/lib/python3.9/site-packages/cwltool/workflow_job.py", line 731, in
shortname(s) for s in scatter if len(cast(Sized, inputobj[s])) == 0
TypeError: object of type 'NoneType' has no len()
```
It should not matter that scatter is empty, because `sort_fastq` step is not executed.
I fixed the problem, using dummy input 'blank_input' in `sort_fastq` with the same input type than `list_fastqs_from_dirs_with_ngs_data/files` that is always not empty. The code with solution looks like that:
```
fastq_file:
source: [list_fastqs_from_dirs_with_ngs_data/files, blank_input]
pickValue: first_non_null
```
and works.
Exactly the same problem also exists in arvados-cwl-runner, which I described here: https://cwl.discourse.group/t/problem-with-when-statements-in-the-workflow-with-nonetype-input-for-scatter/749/1
cwltool version: 3.1.20230127121939
Could you look into it please?
Contributor guide
Assessment
This issue has not been assessed yet.