cockroachdb / cockroachdb/cockroach

storage: MVCCIncrementalIterator ignores intents below StartTime

Open
#173,099 0 comments 0 reactions 0 assignees View on GitHub
A-kv-transactions A-non-blocking-reads branch-master C-bug P-2 T-kv
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Describe the problem**

The MVCCIncrementalIterator ignores intents below the StartTime:

```
metaTimestamp := i.meta.Timestamp.ToTimestamp()
if i.startTime.Less(metaTimestamp) && metaTimestamp.LessEq(i.endTime) {
```

This behaviour is intentional, but it poses at least two problems:

## Potential Bug: ResolveIntent beneath closed timestamp can result in follower-reads skipping an intent they shouldn't skip

The following is an unverified hypothetical timeline:

1. Txn A writes k@ts1.
2. Txn B encounter k and pushes Txn A to ts2
3. Txn B resolves intent on k to ts2.
4. Application of resolve intent from (3) is delayed on some follower.
5. Range for k closes timestamp ts4.
6. Txn B has it's Write timestamp pushed to ts3.
7. Txn B sends a Refresh for `k (ts1, ts3]`. Since ts4 is closed, this can be served by the delayed follower from (4).
8. Refresh erroneous succeeds.

## Problem For VIR

Virtual Intent Resolution aims to reduce the impact of intents on readers and set the groundwork for non-blocking readers by allowing readers to "virtually resolve" intents in their local read snapshot without actually committing a write batch that resolves the intent. In this world, encountered intents will _often_ stay at their original timestamp on disk, meaning that Refresh and RefreshRange would erroneously skip them in the common case.

Jira issue: CRDB-66399

Epic CRDB-63288

Contributor guide

Open the contributing guide

Research direction

Start at MVCCIncrementalIterator and inspect the intent timestamp condition, then trace how Refresh and RefreshRange consume its results. Done means the behavior is specified and the reported follower-read and VIR cases are covered without incorrectly skipping intents.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.