PAIR-code / PAIR-code/deliberate-lab

CI: Codify migrations procedure

Open
#1,022 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
TypeScript
Stars
96
Forks
40
Avg merge
2d 11h
Merged PRs (30d)
20

Description

From time to time, data formats for Deliberate Lab may change. In such cases, existing documents in the Firestore database may need to be bulk migrated to a new format. For example, #939 introduces a functions/src/migrations/migrate-variable-configs.ts file, which converts variable configurations from an older to newer format.

Managing schema migrations such as this is not a new problem. SQL table schemas have the same problem.

We should investigate industry best practices for migrating documents in document datastores like Firebase so that we can choose a strategy for handling these going forward. We should aim to serve both the canonical Deliberate Lab installation as well as third parties running their own instances.

Two concerns are top of mind:

  1. Running hermetic migrations: Building and running migration scripts in a hermetic environment such as Google Cloud Build. This avoids problems of local development setup, which may be in a bad state.
  2. Automatically running migrations: Detecting whether migrations have run, and if not, running them exactly once. This could be rolled into the canonical Google Cloud Build pipeline so that running an "all" deployment automatically performs migrations.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with functions/src/migrations/migrate-variable-configs.ts, introduced by #939, and review the current deployment setup for Google Cloud Build. Research migration practices for Firebase document datastores, focusing on hermetic execution and exactly-once handling. Done means a documented, agreed strategy that supports the canonical installation and third-party instances.

Written by the indexing model from the issue text.

Assessment

Tech stack
firebase, google-cloud, typescript
Domain
ci-cd, cloud, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.