common-workflow-language / common-workflow-language/cwltool

Staging input files in a non-writable sub-directory fails on Linux

Open
#1,042 3 comments 0 reactions 0 assignees View on GitHub
bug help wanted
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

---

## Expected Behavior
Completed run.

## Actual Behavior
Executing on Mac this completes, but on Linux it fails due to trying to delete and not having write permissions for the staged file (discovered by Jenkins, confirmed on a local Ubuntu machine).

Setting the folder to be staged as `writable: true` gets past this error as it avoids calling `os.remove(host_outdir_tgt)`
``

## Workflow Code
Workflow code can be found here:
https://github.com/common-workflow-language/common-workflow-language/pull/820/commits/09a897514c7a9079da8f3cfde858e30f695ef0ae
```
class: CommandLineTool
cwlVersion: v1.0
baseCommand:
- ls
- staged
inputs:
- id: infiles
type: File[]
outputs:
- id: outfile
type: File
outputBinding:
glob: staged/whale.txt
requirements:
- class: InitialWorkDirRequirement
listing:
- entry: >-
${ return { 'class': 'Directory', 'listing': inputs.infiles,
'basename': 'staged'} }
- class: InlineJavascriptRequirement
hints:
- class: DockerRequirement
dockerPull: 'debian:stretch-slim'
```

## Full Traceback
```pytb
Exception while running job
Traceback (most recent call last):
File "/home/bogdan/Documents/cwltool/cwltool/job.py", line 329, in _execute
inplace_update=self.inplace_update)
File "/home/bogdan/Documents/cwltool/cwltool/job.py", line 148, in relink_initialworkdir
os.remove(host_outdir_tgt)
PermissionError: [Errno 13] Permission denied: '/tmp/uue6d20d/staged/whale.txt'
[job stage_array_in_dir.cwl] completed permanentFail
{}
Final process status is permanentFail
```

## Your Environment
* cwltool version: 1.0.20181217162649
Check using ``cwltool --version``

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the linked staging workflow on Linux, then start in cwltool/job.py at relink_initialworkdir and the os.remove(host_outdir_tgt) traceback. Confirm the run completes without the PermissionError while preserving the expected staged output behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, python
Domain
cli, operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.