clockworklabs / clockworklabs/SpacetimeDB
Optimize `SnapshotRepository::compress_older_snapshots` to avoid repeatedly visiting every snapshot
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 25.2k
- Forks
- 1.1k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 46
Description
SnapshotRepository::compress_older_snapshots lists every existing snapshot, forces a list of them into memory, sorts it, then visits each one and checks if it's compressed. In normal operation only the most recent one will not be compressed, so this is a lot of unnecessary work. We only capture a transaction every 1M transactions, though, so it's not exactly a hot path.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at SnapshotRepository::compress_older_snapshots and inspect how it lists, stores, sorts, and visits snapshots. Confirm the existing compression behavior, then measure or trace the operation to ensure the completed change avoids unnecessary repeated snapshot visits without changing which snapshots are compressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100