clockworklabs / clockworklabs/SpacetimeDB
Reimplement: core: Store system tables in commitlog
@kim is already working on this.
Since Apr 14, 2026.
- Dominant language
- Rust
- Stars
- 25.2k
- Forks
- 1.1k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 46
Description
This issue tracks reimplementation of the work from stale PR #1535, which is being closed because it is too out of date to merge directly.
Original PR: https://github.com/clockworklabs/SpacetimeDB/pull/1535
Original author: @kim
Original branch: `kim/st-in-commitlog`
Base branch: `master`
## Original PR summary
Changes the bootstrap sequence, such that the initial system table
schemas and contents are stored in the commitlog.
Opening a database proceeds as follows:
- If a snapshot exists, restore it and replay the history suffix
- Otherwise:
- Create the "seed tables"
st_tableandst_columnsin-memory - Replay the history
- Whenever a table in the reserved system table range is found,
reset the system table schema and layout information in the
page manager, so as to repair incomplete schema information.
- Whenever a table in the reserved system table range is found,
- If the history was empty:
- Create the rest of the system tables (i.e. sans seed tables) in a
transaction. - Commit and persist that transaction.
- Create the rest of the system tables (i.e. sans seed tables) in a
- Create the "seed tables"
Alternatives Considered
The already brittle bootrapping process is now even more brittle. In particular
the various incantations of repair / rebuild routines is hard to follow and easy
to break.
It is also unfortunate that the commitlog will be non-empty in case the
initialization from the user-supplied module fails.
As an alternative, a snapshot could be taken right after the database was fully
initialized, incl. the module. Instead of hashes, this snapshot could contain
their preimages and be inlined into the commitlog.
Restoring a database from such a snapshot requires only the statically known
schemas of the seed tables and a single repair pass.
API and ABI breaking changes
It will not be possible to open a database from a commitlog created by an older
version, because bootstrapping the system tables will only occur if the history
is empty.
Expected complexity level and risk
5 - database bootstrap is highly fragile
## Follow-up
- Reimplement this change in a fresh PR against current `master`.
- Carry forward any still-relevant context from the original PR discussion and review.
- Link the new implementation PR back to the original stale PR for historical context.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.