borgbackup / borgbackup/borg

borg2: improve borg check

Open
#9,696 3 comments 0 reactions 0 assignees View on GitHub
cmd: check
Dominant language
Python
Stars
13.7k
Forks
875
Avg merge
11h 15m
Merged PRs (30d)
192

Description

# borg 1.x

In borg 1.x, there was the repository check and the archives check.

Repository.check fulfilled 2 main purposes:
- do a low-level check of the segment file's entries (crc32-check against corruption)
- IIRC, it always did rebuild the repository index (it had to read/parse all segment entries anyway)
- for client/server mode (ssh: repo with server-side borg process), the repo check ran on the server side!

Archives check:
- read all archives
- check if all chunks referenced from archive metadata stream items are in the index
- with --verify-data, actually read/verify all these chunks

# borg2: quick routine check, no --repair

In borg2 with packs, we should adapt to packs and optimize for server-side processing. There is no borg process on the server side, but borgstore has a generic `Store.hash` method:

Use `Store.hash()` method and compute the sha256 of each pack - it needs to match the pack_id. The REST backend optimizes this call and runs the hash computation on the server without transferring pack data to the client. If it matches, we KNOW that the pack is not accidentally corrupted. Do this for all packs and then we are DONE if --repository-only is given.

When using partial check mode, remember up to which pack id we did it, so the next partial check can start from there.

For a full run (no --repository-only), it would next check the index/ namespace and check each index part also (again: `Store.hash`). If all index parts are ok, iterate over all archives, check if all chunks refer to actually existing packs.

# borg2: something is damaged or missing, we need --repair!
... to be continued ...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.