common-workflow-language / common-workflow-language/common-workflow-language
Proposal: Sets of suggested files
- Dominant language
- Common Workflow Language
- Stars
- 1.5k
- Forks
- 193
- PR merge metrics
- No merged PRs in 30d
Description
### Feature Overview
This feature would allow user to specify multiple sets of "suggested" files for the workflow. Suggested file is a file that will be added to the inputs.json for a specified file input node if available on the file system. If file input is not available in the file system it will stay empty and it will be on the user to set the file and ensure successful execution. Behavior in this case is similar to the default vale on file inputs, but in comparison to default vale, suggested file is not mandatory and if not specified or not available it won't be set on the file input node, but the execution will not be prevented (not a validation).
### Current SBG solution
Current SBG solution allows user to define one file per file input node that will populate value of that input upon draft task creation (FE over dynamically defined inputs.json). Files will be copied (softlink) from publicly available project(s) to the project where user is creating the draft task and will populate input node with the file path to the file (file ID in SBG file system). If the file is already present in the project it will not be copied again.
This feature ensures that production workflows are less user error prone and have higher rate of successful task execution, since the user don't have to worry about which file goes to which input node, especially in case of complex workflows which can have as much two or three dozens of file inputs.
Current sbg:draft2 implementation is done as sbg: extension where the user specifies the sbg file system ID for the file for the particular file input node.
```
---
sbg:x: 692.8573887688864
sbg:y: 411.4287578037877
label: File Input Node 1
sbg:suggestedValue:
- path: SBGFileSystemID
name: SuggestedFile1
class: File
sbg:fileTypes: VCF
id: "#fileinputnode1"
type:
- items: File
```
### Feature impovement
What was identified as a necessary improvement was option for the user to specify multiple sets of suggested files for one workflow. This become evident at the moment when hg38, b38 started to be more and more adopted. At this point user would switch the reference file but not the rest of the database files which because of the standard could be used only with the particular reference file. Also since the hg39, b39 is around the corner, there will be now six possible combinations and the option to only declare and set one set of suggested files. This is why it will be necessary to extend this feature to be able to handle multiple sets of suggested files that user can easily set, and executor, based on the info about this in the app, to prepare automatically inputs.json related to these inputs.
### Proposed solution
```
---
inputs:
- id: "#input1_id"
type:
- File
sbg:suggestedValue:
- id: set1
path: "/path/to/the/file/in/set1.ext"
class: File
name: file1_genome_build_37.ext
- id: set2
path: "/path/to/the/file/in/set2.ext"
class: File
name: file1_genome_build_38.ext
label: First Input Node
- id: "#input2_id"
type:
- File
sbg:suggestedValue:
- id: set1
path: "/another/path/to/the/file/in/set1.ext"
class: File
name: file2_genome_build_37.ext
- id: set2
path: "/another/path/to/the/file/in/set2.ext"
class: File
name: file2_genome_build_38.ext
label: Second Input Node
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.