borgbackup / borgbackup/borg

borg2: avoid doing an unnecessary rebuild of the index

Open
#8,476 6 comments 0 reactions 0 assignees View on GitHub
c: index or hashtable cmd: check cmd: compact
Dominant language
Python
Stars
13.7k
Forks
875
Avg merge
11h 15m
Merged PRs (30d)
192

Description

Note: updated for situation WITH packs.

For high-latency stores (`sftp:` and a lot of what's available via `rclone:`) a full index rebuild needs to list all objects in all packs in the store and there are potentially a lot of objects. This is slow as it needs to scan over each pack file - only borg check --repair-index shall do that.

Most commands will use the existing chunks index. Most commands only add objects, so they will just write an additional partial index to index/sha256(partial_index) with just the new index entries.

For borg create:
- it's not a big problem if the index does not have all objects that exist in the repo. If that happens, borg create will just store something to the repo that's already there.
- it would be a severe problem though if the index would falsely say "we have that object" and borg would not store it to the repo. the archive would then reference a non-existing object.
- similarly it is fatal if the index points to a wrong pack_id

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.