facebookresearch / facebookresearch/flashy
[Bug] Windows error with write_and_rename due to os.rename behavior
- Dominant language
- Python
- Stars
- 118
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
I encountered the following error when running my training script on Windows:
`File "...\.venv\Lib\site-packages\flashy\utils.py", line 54, in write_and_rename os.rename(tmp_path, path)`
`FileExistsError: [WinError 183] Impossible de créer un fichier déjà existant: 'C:\\...\\outputs\\xps\\419014c3\\checkpoint.th.tmp' -> 'C:\\...\\outputs\\xps\\419014c3\\checkpoint.th'`
- This happens when running the code on Windows
- The same code runs correctly on Linux
- The failure occurs during checkpoint saving in `write_and_rename`, called by `solver.commit`
This is due to differences in os.rename behavior between Unix and Windows:
From the [doc](https://docs.python.org/3/library/os.html):
> os.rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None): Rename the file or directory src to dst. If dst exists, the operation will fail with an [OSError](https://docs.python.org/3/library/exceptions.html#OSError) subclass in a number of cases: On Windows, if dst exists a [FileExistsError](https://docs.python.org/3/library/exceptions.html#FileExistsError) is always raised. [...] On Unix, if both are files, dst will be replaced silently if the user has permission. **If you want cross-platform overwriting of the destination, use [replace()](https://docs.python.org/3/library/os.html#os.replace).**
A quick fix would simply be, as suggested above, to replace os.rename with os.replace:
`os.replace(tmp_path, path)`
This ensures atomic replacement behavior on both Windows and Linux.
Contributor guide
Assessment
This issue has not been assessed yet.