PAIR-code / PAIR-code/deliberate-lab
CI: Codify migrations procedure
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 96
- Forks
- 40
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 20
Description
From time to time, data formats for Deliberate Lab may change. In such cases, existing documents in the Firestore database may need to be bulk migrated to a new format. For example, #939 introduces a functions/src/migrations/migrate-variable-configs.ts file, which converts variable configurations from an older to newer format.
Managing schema migrations such as this is not a new problem. SQL table schemas have the same problem.
We should investigate industry best practices for migrating documents in document datastores like Firebase so that we can choose a strategy for handling these going forward. We should aim to serve both the canonical Deliberate Lab installation as well as third parties running their own instances.
Two concerns are top of mind:
- Running hermetic migrations: Building and running migration scripts in a hermetic environment such as Google Cloud Build. This avoids problems of local development setup, which may be in a bad state.
- Automatically running migrations: Detecting whether migrations have run, and if not, running them exactly once. This could be rolled into the canonical Google Cloud Build pipeline so that running an "all" deployment automatically performs migrations.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with functions/src/migrations/migrate-variable-configs.ts, introduced by #939, and review the current deployment setup for Google Cloud Build. Research migration practices for Firebase document datastores, focusing on hermetic execution and exactly-once handling. Done means a documented, agreed strategy that supports the canonical installation and third-party instances.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- firebase, google-cloud, typescript
- Domain
- ci-cd, cloud, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100