cockroachdb / cockroachdb/cockroach
storage: MVCCIncrementalIterator ignores intents below StartTime
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Describe the problem**
The MVCCIncrementalIterator ignores intents below the StartTime:
```
metaTimestamp := i.meta.Timestamp.ToTimestamp()
if i.startTime.Less(metaTimestamp) && metaTimestamp.LessEq(i.endTime) {
```
This behaviour is intentional, but it poses at least two problems:
## Potential Bug: ResolveIntent beneath closed timestamp can result in follower-reads skipping an intent they shouldn't skip
The following is an unverified hypothetical timeline:
1. Txn A writes k@ts1.
2. Txn B encounter k and pushes Txn A to ts2
3. Txn B resolves intent on k to ts2.
4. Application of resolve intent from (3) is delayed on some follower.
5. Range for k closes timestamp ts4.
6. Txn B has it's Write timestamp pushed to ts3.
7. Txn B sends a Refresh for `k (ts1, ts3]`. Since ts4 is closed, this can be served by the delayed follower from (4).
8. Refresh erroneous succeeds.
## Problem For VIR
Virtual Intent Resolution aims to reduce the impact of intents on readers and set the groundwork for non-blocking readers by allowing readers to "virtually resolve" intents in their local read snapshot without actually committing a write batch that resolves the intent. In this world, encountered intents will _often_ stay at their original timestamp on disk, meaning that Refresh and RefreshRange would erroneously skip them in the common case.
Jira issue: CRDB-66399
Epic CRDB-63288
Contributor guide
Research direction
Start at MVCCIncrementalIterator and inspect the intent timestamp condition, then trace how Refresh and RefreshRange consume its results. Done means the behavior is specified and the reported follower-read and VIR cases are covered without incorrectly skipping intents.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100