cockroachdb / cockroachdb/pebble

db: manifest corruption undetected

Open
#4,560 0 comments 0 reactions 0 assignees View on GitHub
A-storage T-storage
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

Today we do not detect corruption of the MANIFEST file. If we encounter an invalid chunk or unexpected EOF while reading a manifest, we consider it to be the logical end of the file and continue on our merry way. This means that bit rot or physical hardware failure that corrupts the manifest will go undetected, and we'll stop replaying the MANIFEST at the point of the corruption.

The write-ahead log uses the same encoding as the manifest but has protection against corruption because it writes 'sync offsets' (see #4331) allowing the reader to read ahead to disambiguate corruption from a torn write at the tail of the manifest. Although the WAL and manifest use the same encoding, they use different writer implementations, and the manifest's does not write sync offsets.

We should adjust the manifest to write the sync offsets so that we can detect corruption within the manifest.

Jira issue: PEBBLE-403

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.