clockworklabs / clockworklabs/SpacetimeDB

Pretty print auto-migration plans

Open
#4,913 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
25.2k
Forks
1.1k
Avg merge
2d 7h
Merged PRs (30d)
46

Description

This issue tracks reimplementing the work from stale PR #2065, which has been closed because it is too far out of date to merge directly.

## Original pull request
- PR #2065: [Pretty print auto-migration plans](https://github.com/clockworklabs/SpacetimeDB/pull/2065)
- Original author: @kazimuth

## What the original PR was trying to do
# Description of Changes

Implements https://github.com/orgs/clockworklabs/projects/22/views/16?filterQuery=migration&pane=issue&itemId=91111218.

While implementing this, I discovered auto-migration plans applied their steps in a nondeterministic order (I was iterating a hashmap and had forgotten to sort.)

This interacted in a particularly bad way with row-level-security policies:

  • Row level security policies are removed and re-added in every automigration, to allow the core crate to re-initialize the relevant queries. (I disagree with this -- it should be an implementation detail of the core crate, not mentioned in auto-migrations -- but I'm not changing it in this PR; see my comments about it in the code.)
  • Since ordering was random, the Add step for the policy could be run before the Remove step.
  • Adding an already existing policy was implemented in an idempotent way by the core crate.
  • So, any auto-migration could nondeterministically drop row-level-security policies!

This is now fixed by simply sorting the steps of an auto-migration in a prescribed way.

API and ABI breaking changes

This is a minor breaking change to the client API since it adds data to the result of an update call.

Expected complexity level and risk

0

Testing

Added a smoketest.
TODO: I want to add to the integration tests of the schema crate as well.

## Closure context
- Closed from the `close-stale-pr-create-issue` stale-PR sweep.

## Reimplementation notes
- Reimplement this work on top of current `master` in a new PR.
- Keep the original PR linked as historical context and as a source of useful implementation ideas where still relevant.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the historical PR #2065 and the current auto-migration implementation on master, then inspect the schema crate's integration tests and the existing smoketest mentioned in the issue. Done means the current implementation pretty-prints migration plans, applies their steps deterministically, and has tests covering the resulting behavior and API changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.