common-workflow-language / common-workflow-language/cwltool
Inconsistent location and name for renamed output of type File when also included in output of type Directory for CLT
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
## Expected Behavior
When an output of type `File` is defined in a `CommandLineTool`, it should be located directly in the specified `--outdir` even if it is also part of another output of type `Directory` (in that case, two copies should exist). I understand that this might not be desired to avoid having two copies of the file. However, even when you want to avoid the duplication, when renaming the output of type `File` by changing its `basename` in an expression, not only the `basename` but also the `location` and `path` should reflect the new `basename` as otherwise the output object provided for the file is inconsistent.
## Actual Behavior
When generating a file within a directory and then tracking both the directory and the file as two separate outputs of one `CommandLineTool` (one of type `Directory` and one of type `File`), the output of type `File` no longer is stored directly within the output directory but only within the directory structure provided by the output of type `Directory` (i.e. only one copy of the file is tracked as output within the directory). This is independent of whether the file is renamed or not. Furthermore, renaming the output of type `File` by changing the `basename` in an `outputEval` expression causes an inconsistent output type with the `basename` being changed but the `location` and `path` (as well as the actual physical location of the file) still corresponding to the old filename.
## Workflow Code
```
class: 'CommandLineTool'
cwlVersion: 'v1.2'
requirements:
- class: 'ShellCommandRequirement'
- class: 'InlineJavascriptRequirement'
baseCommand: ['mkdir', 'test']
inputs:
- id: 'dummy'
type: 'string'
arguments:
- id: 'connect'
valueFrom: '&&'
shellQuote: false
- 'touch'
- 'test/test_file.txt'
outputs:
- id: 'output_dir'
type: 'Directory'
outputBinding:
glob: 'test'
- id: 'output_file'
type: 'File'
outputBinding:
glob: 'test/test_file.txt'
outputEval: |-
${
self[0].basename='renamed.txt';
return self
}
```
with input
```
dummy: 'foobar'
```
provides the following output:
```
...
mkdir test && touch test/test_file.txt
INFO [job mcve_dir_and_rename_file.cwl] completed success
{
"output_dir": {
"location": "file:///test",
"basename": "test",
"class": "Directory",
"listing": [
{
"class": "File",
"location": "file:///test/test_file.txt",
"basename": "test_file.txt",
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size": 0,
"path": "/test/test_file.txt"
}
],
"path": "/test"
},
"output_file": {
"location": "file:///test/test_file.txt",
"basename": "renamed.txt",
"class": "File",
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size": 0,
"path": "/test/test_file.txt"
}
}
INFO Final process status is success
```
When removing the output of type `Directory`, the file is directly in the output dir and renamed as expected (same input):
```
class: 'CommandLineTool'
cwlVersion: 'v1.2'
requirements:
- class: 'ShellCommandRequirement'
- class: 'InlineJavascriptRequirement'
baseCommand: ['mkdir', 'test']
inputs:
- id: 'dummy'
type: 'string'
arguments:
- id: 'connect'
valueFrom: '&&'
shellQuote: false
- 'touch'
- 'test/test_file.txt'
outputs:
- id: 'output_file'
type: 'File'
outputBinding:
glob: 'test/test_file.txt'
outputEval: |-
${
self[0].basename='renamed.txt';
return self
}
```
produces:
```
...
mkdir test && touch test/test_file.txt
INFO [job mcve_rename_file.cwl] completed success
{
"output_file": {
"location": "file:///renamed.txt",
"basename": "renamed.txt",
"class": "File",
"checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709",
"size": 0,
"path": "/renamed.txt"
}
}
INFO Final process status is success
```
## Full Traceback
N/A
## Your Environment
* cwltool version: 3.1.20211014180718 (also tested with 3.1.20220217222804)
Contributor guide
Assessment
This issue has not been assessed yet.