common-workflow-language / common-workflow-language/cwltool
Unable to rename file inside directory by altering the basename
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
## Description
As suggested in https://www.biostars.org/p/384082/ files can be renamed inside a CWL `ExpressionTool` by altering their `basename` property.
This seems to work well for a simple `File` input but appears to not be working (anymore) for the contents of the `listing` of a simple `Directory` input.
### Environment
- Ubuntu 18.04 LTS
- Tested with CWL versions `1.0.20190228155703`, `1.0.20190831161204`, `2.0.20200126090152` and the current version `2.0.20200219182542`
## How to reproduce
### Prerequisites
Move to an empty directory of your choice and execute:
```
mkdir myDir output
touch myDir/oldname.dat
cat << "EOF" > renameFile.cwl
cwlVersion: v1.0
class: ExpressionTool
requirements:
- class: InlineJavascriptRequirement
inputs:
oldFile: File
outputs:
newFile: File
expression: |
${
inputs.oldFile.basename = "newname.dat";
return { "newFile": inputs.oldFile };
}
EOF
cat << "EOF" > renameFirstFileInDir.cwl
cwlVersion: v1.0
class: ExpressionTool
requirements:
- class: InlineJavascriptRequirement
inputs:
oldDir: Directory
outputs:
newDir: Directory
expression: |
${
inputs.oldDir.listing[0].basename = "newname.dat";
return { "newDir": inputs.oldDir };
}
EOF
cd output
```
The directory layout should now look like this:
```
..
├── myOldDir
│ └── oldname.dat
├── output <- current working directory
├── renameFile.cwl
└── renameFirstFileInDir.cwl
```
### Rename plain `File`
```
cwltool ../renameFile.cwl --oldFile ../myDir/oldname.dat
```
results in the expected outcome:
```
../output/
└── newname.dat
```
Works. Great!
Cleanup:
```
rm newname.dat
```
### Rename `File` inside a `Directory`
```
cwltool --debug ../renameFirstFileInDir.cwl --oldDir ../myDir/
```
results in the **_unexpected_** outcome:
```
../output/
└── myDir
└── oldname.dat <-- should now be named "newname.dat"
```
Does not work. File has not been renamed.
Therefore execution also throws an error:
```
ERROR Unhandled error:
[Errno 2] No such file or directory: '/[REDACTED]/output/myDir/newname.dat'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/cwltool/main.py", line 1099, in main
tool, initialized_job_order_object, runtimeContext, logger=_logger
File "/usr/local/lib/python3.6/dist-packages/cwltool/executors.py", line 43, in __call__
return self.execute(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cwltool/executors.py", line 132, in execute
path_mapper=runtime_context.path_mapper,
File "/usr/local/lib/python3.6/dist-packages/cwltool/process.py", line 411, in relocateOutputs
outputObj, ("File",), functools.partial(compute_checksums, fs_access)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 231, in visit_class
visit_class(rec[d], cls, op)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 231, in visit_class
visit_class(rec[d], cls, op)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 234, in visit_class
visit_class(d, cls, op)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 229, in visit_class
op(rec)
File "/usr/local/lib/python3.6/dist-packages/cwltool/process.py", line 1207, in compute_checksums
with fs_access.open(fileobj["location"], "rb") as f:
File "/usr/local/lib/python3.6/dist-packages/cwltool/stdfsaccess.py", line 41, in open
return open(self._abs(fn), mode)
FileNotFoundError: [Errno 2] No such file or directory: '/[REDACTED]/output/myDir/newname.dat'
```
Contributor guide
Research direction
Reproduce the issue with the two ExpressionTool files and the cwltool commands shown. Start by reading cwltool/process.py around relocateOutputs, then follow utils.py visit_class and stdfsaccess.py; done means a basename change within a Directory listing is handled consistently and the output completes without the missing-file error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100