borg create: try to speed up unchanged file processing
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 875
- Avg merge
- 11h 15m
- Merged PRs (30d)
- 192
Description
Early borg used to work filepath-based to deal with the source files (stat, open, read, xattrs, acls, fsflags).
That was problematic due to race conditions and thus was changed in borg 1.2 to use `open()` to get a file descriptor and then work with the fd everywhere possible, so we can be sure to always deal with the same fs object independently of its path.
But: fs api calls and especially `open()` can be rather slow for some filesystems, like network filesystems.
So, for an **unchanged** file (files cache hit), it currently does:
- st1 = stat(path)
- fd = open(path)
- st2 = fstat(fd)
- check st1 against st2
- here it notices (by checking against the files cache contents) that the file is unchanged and decides not to read the files content, but reuse the chunkids from files cache.
- reads xattrs, except when --noxattrs is given
- reads acls, except when --noacls is given
- reads fsflags (bsdflags), except when --noflags is given
- creates a new archive item
Review the code if it can be modified for the unchanged file case, so that the `open` and `fstat` call is not needed, without causing issues like re-introducing races.
Contributor guide
Assessment
This issue has not been assessed yet.