Partial write is actually O(N^2) for non copy-on-write FS
Open
type:support
- Dominant language
- Python
- Stars
- 535
- Forks
- 101
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 23
Description
Seems like that due to atomicity we create a recursive copy of the checkpoint on every new entry.
https://github.com/google/orbax/blob/main/checkpoint/orbax/checkpoint/_src/path/snapshot/snapshot.py#L72
For the regular NFS/S3 that doesn't support copy-on-write, this actually leads to O(N^2) traffic usage, because we make a full copy of existing checkpoint on each update. Is there a way to avoid that, even at the cost of disabling the atomicity? I don't really need it in my case by still want to use partial writes.
Contributor guide
Assessment
This issue has not been assessed yet.