cockroachdb / cockroachdb/pebble
db: manifest corruption undetected
- Dominant language
- Go
- Stars
- 6k
- Forks
- 584
- Avg merge
- 16h 35m
- Merged PRs (30d)
- 5
Description
Today we do not detect corruption of the MANIFEST file. If we encounter an invalid chunk or unexpected EOF while reading a manifest, we consider it to be the logical end of the file and continue on our merry way. This means that bit rot or physical hardware failure that corrupts the manifest will go undetected, and we'll stop replaying the MANIFEST at the point of the corruption.
The write-ahead log uses the same encoding as the manifest but has protection against corruption because it writes 'sync offsets' (see #4331) allowing the reader to read ahead to disambiguate corruption from a torn write at the tail of the manifest. Although the WAL and manifest use the same encoding, they use different writer implementations, and the manifest's does not write sync offsets.
We should adjust the manifest to write the sync offsets so that we can detect corruption within the manifest.
Jira issue: PEBBLE-403
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.