blockstore: OpenReadWrite should use atomic writes
- Dominant language
- Go
- Stars
- 175
- Forks
- 54
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 4
Description
Right now, OpenReadWrite writes directly to the destination file. This has multiple problems:
* If we encounter some error mid-finalize, we may leave a corrupted file
* If we encounter a write/flush error while finalizing, we may leave a partial file
* If we're resuming on an existing file and writing in-place, we may corrupt the user's data (e.g. https://github.com/ipld/go-car/issues/247)
It would be much saner overall to instead write to a temporary copy next to the destination file (such as `foo.car.tmp` for `foo.car`), and once `Finalize` has finished with no error, we do a rename that should basically never fail.
This also means that we can teach OpenReadWrite to always remove `foo.car.tmp` when it's discarded, to ensure we don't leave unfinished business behind.
The method described above should be doable with just a bit of `os` and `io` glue. We could use https://pkg.go.dev/github.com/google/renameio#TempFile, though I think pulling in a library for this use case is perhaps overkill. We want to prevent common errors, and we're not *that* concerned with atomicity between multiple processes - users should not be writing to the same CAR file concurrently.
cc @masih @willscott
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.