common-workflow-language / common-workflow-language/cwltool
CWL fails to reconginze the existing files to generate new outputs using inplace editing
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
I am using CWL to generate some output based on certain existing output files in outdir.
---
## Expected Behavior
The CWL should take in the existing output files in the designated output directory and generate new outputs. The designated output dir consists of some .pickle files. The CWL should take them and generate single merged hdf5 file.
## Actual Behavior
Instead CWL scripts starts from the scratch. Which is processing wise time consuming
## Workflow Code
```
cwlVersion: v1.0
class: CommandLineTool
doc: Spladder
baseCommand: [python,-m,spladder.spladder]
hints:
cwltool:InplaceUpdateRequirement:
inplaceUpdate: true
requirements:
EnvVarRequirement:
envDef:
PYTHONPATH: /cluster/home/aalva/spladder/python
InlineJavascriptRequirement: {}
InitialWorkDirRequirement:
listing:
- entry: $(inputs.spladder_dir)
writable: true
inputs:
spladder_gtf:
type: File
inputBinding:
position: 3
prefix: -a
spladder_bams:
type: File[]
inputBinding:
position: 1
prefix: -b
itemSeparator: ","
secondaryFiles: .bai
spladder_dir:
type: Directory
inputBinding:
position: 2
prefix: -o
spladder_phase2:
type: string
inputBinding:
position: 4
prefix: -T
spladder_merge_graphs:
type: string
inputBinding:
position: 5
prefix: -M
spladder_qmode:
type: string
inputBinding:
position: 7
prefix: --qmode
spladder_quantify_graph:
type: string
inputBinding:
position: 6
prefix: --quantify_graph
outputs:
spladder_out:
type: Directory
outputBinding:
glob: $(inputs.spladder_dir.basename)
$namespaces:
cwltool: http://commonwl.org/cwltool#
```
## Full Traceback
```
Strted to run from scratch in temp file
```
## Your Environment
* cwltool version: v1.0
I ran it as `cwltool --enable-ext spl.cwl sp.yml`
Contributor guide
Research direction
Start by reviewing the provided CWL workflow, especially cwltool:InplaceUpdateRequirement and InitialWorkDirRequirement, then reproduce it with `cwltool --enable-ext spl.cwl sp.yml` using the reported cwltool v1.0 environment. Determine why the existing `.pickle` files in the designated output directory are not reused; done means the workflow recognizes them and produces the merged HDF5 output without restarting from scratch.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100