cockroachdb / cockroachdb/pebble

snapshot-backed batches

Open
#1,416 10 comments 3 reactions 0 assignees View on GitHub
C-question O-community
Dominant language
Go
Stars
6k
Forks
584
Avg merge
16h 35m
Merged PRs (30d)
5

Description

Is there a reason why the current API doesn't allow for creating a batch that uses a snapshot for a point-in-time view of the database?

It looks simple enough to add - *too* simple actually, so I figure'd it'd be good to ask before making a PR. https://github.com/cockroachdb/pebble/blob/e2a8aef27e1b170d78f466fb97fda036a693abab/batch.go#L418

The use-case is for supporting a number of concurrent processing engines in a blockchain system. For instance, there is a "mempool" of uncommitted messages that should be processed using the most recent committed state, but does not commit changes itself; and there is another process which executes messages and commits state according to a consensus protocol.

The mempool processing could (very vaguely) look something like this:

1. create snapshot
2. create snapshot-backed batch
3. for each new message:
3.1. push savepoint
3.2. process message
3.3. pop savepoint if transaction succeeds, rollback if transaction fails
3.4. if new state has been committed via consensus, release snapshot and start at **1.**

Jira issue: PEBBLE-115

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with batch.go around line 418 and inspect how the current batch API relates to snapshots. Validate the proposed lifecycle against the mempool and consensus workflow, then define tests for point-in-time reads, savepoints, rollback, and snapshot release.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.