common-workflow-language / common-workflow-language/cwltool
OSError: [Errno 18] Invalid cross-device link error in move os.rename(src, real_dst)
- Dominant language
- Python
- Stars
- 376
- Forks
- 255
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 12
Description
Hi, I use cwltool to run a pipeline and I sometimes have an error when I run it. The error is : `OSError: [Errno 18] Invalid cross-device link` and `File "/path/to/.conda/envs/name_env/lib/python3.7/shutil.py", line 566, in move os.rename(src, real_dst) `
I think the error is the same as here and the problem is the using of `os.rename` instead of `shutil.move().`
This error appears at the end of the pipeline when cwltool moves temp files into local.
Have you ever encountered this error ? If yes, is there a way to troubleshoot it ?
---
## Expected Behavior
no error
## Actual Behavior
`OSError: [Errno 18] Invalid cross-device link` and `File "/path/to/.conda/envs/name_env/lib/python3.7/shutil.py", line 566, in move os.rename(src, real_dst) `
## Full Traceback
```pytb
^[[1;30mERROR^[[0m ^[[31mUnhandled error:
[Errno 2] No such file or directory: '/path/bins_directory/reads_full_pe_A.fq_1_bins_metabat.1.fa'^[[0m
Traceback (most recent call last):
File "/path/to/.conda/envs/env_conda/lib/python3.7/shutil.py", line 566, in move
os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/tmp/npmifkcp/MetaBat/reads_full_pe_A.fq_1_bins_metabat.1.fa' -> '/path/bins_directory/reads_full_pe_A.fq_1_bins_metabat.1.fa'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/.local/lib/python3.7/site-packages/cwltool/main.py", line 1249, in main
tool, initialized_job_order_object, runtimeContext, logger=_logger
File "/home/.local/lib/python3.7/site-packages/cwltool/executors.py", line 59, in __call__
return self.execute(process, job_order_object, runtime_context, logger)
File "/home/.local/lib/python3.7/site-packages/cwltool/executors.py", line 162, in execute
path_mapper=runtime_context.path_mapper,
File "/home/.local/lib/python3.7/site-packages/cwltool/process.py", line 400, in relocateOutputs
stage_files(pm, stage_func=_relocate, symlink=False, fix_conflicts=True)
File "/home/.local/lib/python3.7/site-packages/cwltool/process.py", line 296, in stage_files
stage_func(entry.resolved, entry.target)
File "/home/.local/lib/python3.7/site-packages/cwltool/process.py", line 371, in _relocate
_relocate(dir_entry.path, fs_access.join(dst, dir_entry.name))
File "/home/.local/lib/python3.7/site-packages/cwltool/process.py", line 373, in _relocate
shutil.move(src, dst)
File "/home/.conda/envs/name_env/lib/python3.7/shutil.py", line 580, in move
copy_function(src, real_dst)
File "/home/.conda/envs/name_env/lib/python3.7/shutil.py", line 266, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/home/.conda/envs/name_env/lib/python3.7/shutil.py", line 121, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/path/bins_directory/reads_full_pe_A.fq_1_bins_metabat.1.fa'
```
## Your Environment
* cwltool version: 3.1.20210816212154
Contributor guide
Assessment
This issue has not been assessed yet.