common-workflow-language / common-workflow-language/cwltool

Feature request: set secondaryFiles in workflow step input

Open
#538 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

I would like to be able to set `secondaryFiles` for inputs of workflows steps. For example, GATK requires the following reference genome files: FASTA, FAI and DICT. The last two files are generated by different programs. To make a workflow fully reproducible, I think the same workflow should create those files before running the step that requires all files related with reference genome.

## Expected Behavior
To be able to set `secondaryFiles` for inputs of workflow steps.

## Actual Behavior
Related files should be set in job file or pass them using additional params.

## Workflow Code
``` yml
steps:
create_sequence_dictionary:
run: https://raw.githubusercontent.com/labbcb/tool-picard/master/picard-createsequencedictionary.cwl
in:
reference: ref
out: [output_file]
faidx:
run: https://raw.githubusercontent.com/labbcb/tool-samtools/master/samtools-faidx.cwl
in:
ref_fasta: ref
out: [index_file]

# Align FASTQ files, build BAM index ...

base_recalibrator:
run: https://raw.githubusercontent.com/labbcb/tool-gatk/master/gatk-baserecalibrator.cwl
scatter: [input_file]
in:
input_file: bam_index/output
reference_sequence: ref
# faidx/index_file and create_sequence_dictionary/output_file should be
# secondaryFiles of reference_sequence
reference_index: faidx/index_file
reference_dictionary: create_sequence_dictionary/output_file
intervals: intervals
knownSites: knownSites
out:
valueFrom: $(inputs.input_file.basename).recal_data.table
out: [output]
```

Input example:

``` yml
ref:
class: File
path: Homo_sapiens_assembly38.fasta
intervals:
class: File
path: targets_hg38.interval_list
knownSites:
- class: File
path: dbsnp_144.hg38.vcf.gz
secondaryFiles:
- class: File
path: dbsnp_144.hg38.vcf.gz.tbi
- class: File
path: Mills_and_1000G_gold_standard.indels.hg38.vcf.gz
secondaryFiles:
- class: File
path: Mills_and_1000G_gold_standard.indels.hg38.vcf.gz.tbi
```

## Your Environment

* cwltool version: 1.0.20170817131858

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.