cockroachdb / cockroachdb/pebble
snapshot-backed batches
- Dominant language
- Go
- Stars
- 6k
- Forks
- 584
- Avg merge
- 16h 35m
- Merged PRs (30d)
- 5
Description
Is there a reason why the current API doesn't allow for creating a batch that uses a snapshot for a point-in-time view of the database?
It looks simple enough to add - *too* simple actually, so I figure'd it'd be good to ask before making a PR. https://github.com/cockroachdb/pebble/blob/e2a8aef27e1b170d78f466fb97fda036a693abab/batch.go#L418
The use-case is for supporting a number of concurrent processing engines in a blockchain system. For instance, there is a "mempool" of uncommitted messages that should be processed using the most recent committed state, but does not commit changes itself; and there is another process which executes messages and commits state according to a consensus protocol.
The mempool processing could (very vaguely) look something like this:
1. create snapshot
2. create snapshot-backed batch
3. for each new message:
3.1. push savepoint
3.2. process message
3.3. pop savepoint if transaction succeeds, rollback if transaction fails
3.4. if new state has been committed via consensus, release snapshot and start at **1.**
Jira issue: PEBBLE-115
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with batch.go around line 418 and inspect how the current batch API relates to snapshots. Validate the proposed lifecycle against the mempool and consensus workflow, then define tests for point-in-time reads, savepoints, rollback, and snapshot release.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100