cockroachdb / cockroachdb/cockroach
storage: rewrite CheckSSTConflicts
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
The current implementation of the `storage.CheckSSTConflicts` function is very complicated, and it's very difficult to reason about its correctness. We've uncovered many bugs (eg, #99566) in its implementation as a result, and today largely rely on randomized testing (from KV nemesis and the unit test in #98408) to ensure correctness. We should rewrite and simplify it.
Some ideas:
* @itsbilal suggested a four-iterator approach. We'd use 1 range key iterator and 1 point iterator for both the engine and the sstable.
* We could implement a "tandem" iterator that keeps the engine iterator positioned appropriately as the sstable iterator moves across the keyspace. There are some subtleties in what "positioned appropriately" means (eg, the engine iterator may need to read _before_ the sstable iterator's position in order to observe a range key). Separating out the mechanics of keeping the two iterators in sync should make the actual conflict checking more tractable.
Jira issue: CRDB-26007
Contributor guide
Research direction
Start with storage.CheckSSTConflicts, then read the KV nemesis coverage and the unit test from #98408. Review the failure described in #99566 and the proposed four-iterator or tandem-iterator approach. Done means a simpler implementation whose conflict-checking behavior remains correct under the existing randomized and unit tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100