OpenFreeEnergy / OpenFreeEnergy/openfe
Nonspecific rmtree failure when accidentally resuming a job twice
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 331
- Forks
- 56
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 13
Description
If you try to quickrun resume the same transformation twice, you can end up in a situation where two parallel attempts at completing the job are happening.
When that happens, one of the jobs can eventually fail with this kind of stack trace:
Traceback (most recent call last):
File "/home/ialibay/software/mambaforge/install/envs/openfe/bin/openfe", line 6, in <module>
sys.exit(main())
~~~~^^
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/site-packages/click/core.py", line 1485, in __call__
return self.main(*args, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/site-packages/click/core.py", line 1406, in main
rv = self.invoke(ctx)
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/site-packages/click/core.py", line 1873, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/site-packages/click/core.py", line 1269, in invoke
return ctx.invoke(self.callback, **ctx.params)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/site-packages/click/core.py", line 824, in invoke
return callback(*args, **kwargs)
File "/home/ialibay/github/openfe/src/openfecli/utils.py", line 116, in wrapper
result = function(*args, **kwargs)
File "/home/ialibay/github/openfe/src/openfecli/commands/quickrun.py", line 160, in quickrun
dagresult = execute_DAG(
dag,
...<5 lines>...
n_retries=2,
)
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/site-packages/gufe/protocols/protocoldag.py", line 526, in execute_DAG
shutil.rmtree(scratch)
~~~~~~~~~~~~~^^^^^^^^^
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/shutil.py", line 763, in rmtree
_rmtree_safe_fd(stack, onexc)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/shutil.py", line 704, in _rmtree_safe_fd
onexc(func, path, err)
~~~~~^^^^^^^^^^^^^^^^^
File "/home/ialibay/software/mambaforge/install/envs/openfe/lib/python3.13/shutil.py", line 665, in _rmtree_safe_fd
orig_st = os.lstat(name, dir_fd=dirfd)
FileNotFoundError: [Errno 2] No such file or directory: PosixPath('results/test/scratch_HybridTopologyMultiStateSimulationUnit-d254ea3d8d0a495bb69ed7c57c219520_attempt_0')
This is probably because the other runner has already deleted the scratch repository. It may make sense to capture that error and provide users with a clearer error.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at src/openfecli/commands/quickrun.py around line 160, then follow the execute_DAG call shown in the traceback into gufe's protocoldag.py. Reproduce a duplicated quickrun resume if possible and inspect how scratch cleanup reaches shutil.rmtree. Done means the duplicate-resume failure reports a clearer user-facing error instead of the nonspecific FileNotFoundError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100