common-workflow-language / common-workflow-language/cwltool

Unable to rename file inside directory by altering the basename

Open
#1,254 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

## Description

As suggested in https://www.biostars.org/p/384082/ files can be renamed inside a CWL `ExpressionTool` by altering their `basename` property.
This seems to work well for a simple `File` input but appears to not be working (anymore) for the contents of the `listing` of a simple `Directory` input.

### Environment

- Ubuntu 18.04 LTS
- Tested with CWL versions `1.0.20190228155703`, `1.0.20190831161204`, `2.0.20200126090152` and the current version `2.0.20200219182542`

## How to reproduce
### Prerequisites

Move to an empty directory of your choice and execute:
```
mkdir myDir output
touch myDir/oldname.dat

cat << "EOF" > renameFile.cwl
cwlVersion: v1.0
class: ExpressionTool
requirements:
- class: InlineJavascriptRequirement
inputs:
oldFile: File
outputs:
newFile: File
expression: |
${
inputs.oldFile.basename = "newname.dat";
return { "newFile": inputs.oldFile };
}
EOF

cat << "EOF" > renameFirstFileInDir.cwl
cwlVersion: v1.0
class: ExpressionTool
requirements:
- class: InlineJavascriptRequirement
inputs:
oldDir: Directory
outputs:
newDir: Directory
expression: |
${
inputs.oldDir.listing[0].basename = "newname.dat";
return { "newDir": inputs.oldDir };
}
EOF

cd output
```

The directory layout should now look like this:
```
..
├── myOldDir
│   └── oldname.dat
├── output <- current working directory
├── renameFile.cwl
└── renameFirstFileInDir.cwl
```

### Rename plain `File`

```
cwltool ../renameFile.cwl --oldFile ../myDir/oldname.dat
```
results in the expected outcome:
```
../output/
└── newname.dat
```
Works. Great!

Cleanup:
```
rm newname.dat
```

### Rename `File` inside a `Directory`

```
cwltool --debug ../renameFirstFileInDir.cwl --oldDir ../myDir/
```
results in the **_unexpected_** outcome:
```
../output/
└── myDir
└── oldname.dat <-- should now be named "newname.dat"
```
Does not work. File has not been renamed.
Therefore execution also throws an error:
```
ERROR Unhandled error:
[Errno 2] No such file or directory: '/[REDACTED]/output/myDir/newname.dat'
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/cwltool/main.py", line 1099, in main
tool, initialized_job_order_object, runtimeContext, logger=_logger
File "/usr/local/lib/python3.6/dist-packages/cwltool/executors.py", line 43, in __call__
return self.execute(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/cwltool/executors.py", line 132, in execute
path_mapper=runtime_context.path_mapper,
File "/usr/local/lib/python3.6/dist-packages/cwltool/process.py", line 411, in relocateOutputs
outputObj, ("File",), functools.partial(compute_checksums, fs_access)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 231, in visit_class
visit_class(rec[d], cls, op)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 231, in visit_class
visit_class(rec[d], cls, op)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 234, in visit_class
visit_class(d, cls, op)
File "/usr/local/lib/python3.6/dist-packages/cwltool/utils.py", line 229, in visit_class
op(rec)
File "/usr/local/lib/python3.6/dist-packages/cwltool/process.py", line 1207, in compute_checksums
with fs_access.open(fileobj["location"], "rb") as f:
File "/usr/local/lib/python3.6/dist-packages/cwltool/stdfsaccess.py", line 41, in open
return open(self._abs(fn), mode)
FileNotFoundError: [Errno 2] No such file or directory: '/[REDACTED]/output/myDir/newname.dat'
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the two ExpressionTool files and the cwltool commands shown. Start by reading cwltool/process.py around relocateOutputs, then follow utils.py visit_class and stdfsaccess.py; done means a basename change within a Directory listing is handled consistently and the output completes without the missing-file error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.