borg2: improve dealing with repo corruption
- Dominant language
- Python
- Stars
- 13.7k
- Forks
- 875
- Avg merge
- 11h 15m
- Merged PRs (30d)
- 192
Description
This is mostly a "please document repo repair a little better" - ticket.
Well, I did something stupid and corrupted some chunks of one of my borg2 repos. (dont ``dd if=/dev/zero of=/dev/sd...`` the wrong device 🙈)
Anyway.
After running ``borg2 check --repair -r /bkp/...`` it deleted the corrupted chunks.
During the next backup it re-created most of the corrupted chunks, so now I am at 2 corrupted chunks, one of which I definitely can never get back. (As it was an encrypted SeedVault-Android backup chunk)
With subsequent runs of ``borg2 check -r /bkp/...`` it will always print the Files affected:
```
[...]: srv/mattermost/server/prepackaged_plugins/mattermost-plugin-msteams-v2.0.3-linux-amd64.tar.gz: Missing file chunk detected (Byte 13617517-15441730, Chunk 160e59edd9a5aeafae625ee9a65139544005c036be61b18136f913ece155dc3a).
[...]: srv/mattermost/server/prepackaged_plugins/mattermost-plugin-msteams-v2.0.3-linux-amd64.tar.gz: Missing file chunk detected (Byte 13617517-15441730, Chunk 160e59edd9a5aeafae625ee9a65139544005c036be61b18136f913ece155dc3a).
[...]: srv/mattermost/server/prepackaged_plugins/mattermost-plugin-msteams-v2.0.3-linux-amd64.tar.gz: Missing file chunk detected (Byte 13617517-15441730, Chunk 160e59edd9a5aeafae625ee9a65139544005c036be61b18136f913ece155dc3a).
[...]: srv/mattermost/server/prepackaged_plugins/mattermost-plugin-msteams-v2.0.3-linux-amd64.tar.gz: Missing file chunk detected (Byte 13617517-15441730, Chunk 160e59edd9a5aeafae625ee9a65139544005c036be61b18136f913ece155dc3a).
[...]: srv/nextcloud/data/[...]/files/.SeedVaultAndroidBackup/3634a43e813679618678283efc753829f884907d24bf8a944d94b463f1e33e79/15/15fc8d419949452ed06b382cda2e09074526ce1c3fbfe47ee43f21e23013ab94: Missing file chunk detected (Byte 0-786493, Chunk 15fc8d419949452ed06b382cda2e09074526ce1c3fbfe47ee43f21e23013ab94).
```
**I dont particulaly care about these files and would like to acknowledge their corruption and ignore them in future.**
Currently borg2 compact exits with exitcode=2 and the message ``Repository has 2 missing objects!`` after successfully completing its run. This causes me to get an error-eMail.
So... what would be the next steps for me to take?
+ I dont want to ignore exitcode=2 as that could also mean ssh connection issues or other important things.
+ I dont want to wait for the archives to rotate away ... this would be more than a year of daily error eMails.
+ I dont want to delete the whole archives ... the other files are seemingly fine.
Can I somehow mark these files as unimporatant?
Can I delete these files from the corresponding archives? (its a quite big repo; I fear recreate would take forever; or is ``--exclude`` faster than recompress?)
Also please document that in the help section for borg check. It currently doesnt even mention that corrupted chunks will get recreated during the next borg2 create, if the original files still exist. What is also not clear to me: Did I actually need a second ``borg2 check --repair`` after borg2 create?
And please add something like "considerations to fix your repo corruption", with ideas that people might want to apply to reach some post-corruption state.
Contributor guide
Research direction
Start with the `borg2 check --repair`, `borg2 create`, and `borg2 compact` behavior described in the issue, then locate the help section for `borg check`. Document how corruption and recreated chunks are handled, explain the available post-corruption recovery choices, and clarify the meaning of the reported exit code so the requested guidance is complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100