facebookresearch / facebookresearch/flashy

[Bug] Windows error with write_and_rename due to os.rename behavior

Open
#3 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
118
Forks
6
PR merge metrics
No merged PRs in 30d

Description

I encountered the following error when running my training script on Windows:

`File "...\.venv\Lib\site-packages\flashy\utils.py", line 54, in write_and_rename os.rename(tmp_path, path)`
`FileExistsError: [WinError 183] Impossible de créer un fichier déjà existant: 'C:\\...\\outputs\\xps\\419014c3\\checkpoint.th.tmp' -> 'C:\\...\\outputs\\xps\\419014c3\\checkpoint.th'`

- This happens when running the code on Windows
- The same code runs correctly on Linux
- The failure occurs during checkpoint saving in `write_and_rename`, called by `solver.commit`

This is due to differences in os.rename behavior between Unix and Windows:

From the [doc](https://docs.python.org/3/library/os.html):

> os.rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None): Rename the file or directory src to dst. If dst exists, the operation will fail with an [OSError](https://docs.python.org/3/library/exceptions.html#OSError) subclass in a number of cases: On Windows, if dst exists a [FileExistsError](https://docs.python.org/3/library/exceptions.html#FileExistsError) is always raised. [...] On Unix, if both are files, dst will be replaced silently if the user has permission. **If you want cross-platform overwriting of the destination, use [replace()](https://docs.python.org/3/library/os.html#os.replace).**

A quick fix would simply be, as suggested above, to replace os.rename with os.replace:
`os.replace(tmp_path, path)`

This ensures atomic replacement behavior on both Windows and Linux.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.