common-workflow-language / common-workflow-language/cwltool

shutil.rmtree fails in M2 mac

Open
#1,913 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

## How to reproduce
Run the following commands.(test.fa is attatched below.)
```console
curl -SOL https://busco-data.ezlab.org/v4/data/lineages/insecta_odb10.2020-09-10.tar.gz
tar xf insecta_odb10.2020-09-10.tar.gz

cwltool bug.cwl --busco_db insecta_odb10 --input_fasta test.fa
```

## Expected Behavior
It finishes without errors.

## Actual Behavior
It fails with the following messages.
It seems that BUSCO successfully finished but cwltool fails cleaning up the temporal directory.
```pytb
cwltool tools/bug.cwl --busco_db insecta_odb10 --input_fasta test.fa
...
INFO [job bug.cwl] Max memory used: 2369MiB
ERROR Exception while running job
Traceback (most recent call last):
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/site-packages/cwltool/job.py", line 373, in _execute
relink_initialworkdir(
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/site-packages/cwltool/job.py", line 103, in relink_initialworkdir
shutil.rmtree(host_outdir_tgt)
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/shutil.py", line 738, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/shutil.py", line 736, in rmtree
os.rmdir(path, dir_fd=dir_fd)
PermissionError: [Errno 13] Permission denied: '/private/tmp/docker_tmp379y2o69/insecta_odb10'
WARNING [job bug.cwl] completed permanentFail
{}WARNING Final process status is permanentFail
```

## Workflow Code
```yaml
#!/usr/bin/env cwl-runner
cwlVersion: v1.2
class: CommandLineTool

baseCommand: [ busco]
arguments: [
-m, genome,
-o, tmp,
--offline,
-i, $(inputs.input_fasta),
-l, $(inputs.busco_db)
]

requirements:
InitialWorkDirRequirement:
listing:
- entry: $(inputs.busco_db)
writable: true
InplaceUpdateRequirement:
inplaceUpdate: true
DockerRequirement:
dockerPull: "quay.io/biocontainers/busco:5.5.0--pyhdfd78af_0"
inputs:
input_fasta:
type: File
busco_db:
type: Directory

outputs: []

```

## Full Traceback
I confirmed --debug shows the same messages.
```pytb
cwltool tools/bug.cwl --busco_db insecta_odb10 --input_fasta test.fa
...
INFO [job bug.cwl] Max memory used: 2369MiB
ERROR Exception while running job
Traceback (most recent call last):
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/site-packages/cwltool/job.py", line 373, in _execute
relink_initialworkdir(
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/site-packages/cwltool/job.py", line 103, in relink_initialworkdir
shutil.rmtree(host_outdir_tgt)
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/shutil.py", line 738, in rmtree
onerror(os.rmdir, path, sys.exc_info())
File "/Users/ohnuki/miniconda3/envs/cwl/lib/python3.11/shutil.py", line 736, in rmtree
os.rmdir(path, dir_fd=dir_fd)
PermissionError: [Errno 13] Permission denied: '/private/tmp/docker_tmp379y2o69/insecta_odb10'
WARNING [job bug.cwl] completed permanentFail
{}WARNING Final process status is permanentFail
```

## Your Environment
* cwltool version:
Check using ``cwltool --version``
```console
cwltool --version
/Users/ohnuki/miniconda3/envs/cwl/bin/cwltool 3.1.20230906142556
```

I confirmed it happens in M2 max Mac(macOS 13.6) but does not happen in M1 Mac(macOS 13.6 and cwltool 3.1.20220830195442) and Ubuntu-22.04(intel and cwltool 3.1.20220913185150).

[test.fa.zip](https://github.com/common-workflow-language/cwltool/files/12739517/test.fa.zip)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.