backstage / backstage/backstage
🚀 Feature: Allow a way for data migrations to be rolled back when downgrading backstage
- Dominant language
- TypeScript
- Stars
- 34.4k
- Forks
- 7.6k
- Avg merge
- 8h 57m
- Merged PRs (30d)
- 50
Description
### 🔖 Feature description
Hi,
The `backstage-cli` should be updated to allow data migrations to be rolled back -- ideally, this would be an implicit process ( i.e. if you try to downgrade backstage from version `B` to version `A`, then data migrations in `B` but not `A` should be rolled back). However if this is too ambiguous, it would still be useful to add commands to `backstage-cli` to allow this in a more explicit process.
### 🎤 Context
🧵 [Discord Thread](https://discord.com/channels/687207715902193673/1195419527362658344) where it was suggested for me to submit a new issue.
Recently we deployed a new version of Backstage to a pre-production environment which included multiple data migrations ( e.g. [20230428155633_sessions.js](https://github.com/backstage/backstage/blob/v1.18.4/plugins/auth-backend/migrations/20230428155633_sessions.js) ). This process went well and the migrations were applied to the database correctly.
Later, we decided to rollback to the previous version. however we encountered a startup failure on the backend with the following error:
```
Backend failed to start up Error: The migration directory is corrupt, the following files are missing: 20230428155633_sessions.js
```
I understand _why_ this is happening -- when we updated to the new version, new rows were added to `knex_migrations` for the new migrations, and those files do not exist in the previous version.
This made rollback difficult. We ended up solving this by doing the following for each missing migration:
1. Connect to our DB and manually run logic in the `down` method of the migration ( e.g. [20230428155633_sessions.js down method](https://github.com/backstage/backstage/blob/v1.18.4/plugins/auth-backend/migrations/20230428155633_sessions.js)
2. Remove the corresponding row from the `knex_migrations` table
3. Retry
This worked but isn't ideal. Moreover, there isn't an easy way to know _which_ migrations are part of a new release. The error at startup only listed a single migration failure, and after solving this it would fail on the next missing migration. It would be really useful if the [Backstage Upgrade Helper](https://backstage.github.io/upgrade-helper/) could include data migrations between the two version as well, or a `backstage-cli` command to do something similar.
For context, we are running in a Kubernetes environment so it isn't uncommon for us to create pods with newer versions of backstage for testing purposes in our lower environments.
### ✌️ Possible Implementation
The migration files all look to include an exported `down` method for the express purpose of undoing a migration, so `backstage-cli` could hook into this the same way it must for the `up` method when it _applies_ the migration. Also, when comparing two versions, a diff of the migrations can be found simply by looking at the `migrations` path of each plugin and comparing the two. While this can be done manually, it is extremely tedious and error prone.
### 👀 Have you spent some time to check if this feature request has been raised before?
- [X] I checked and didn't find similar issue
### 🏢 Have you read the Code of Conduct?
- [X] I have read the [Code of Conduct](https://github.com/backstage/backstage/blob/master/CODE_OF_CONDUCT.md)
### Are you willing to submit PR?
None
Contributor guide
Research direction
Start by inspecting backstage-cli's existing migration handling and the plugin migrations paths, including the referenced auth-backend migration and its down method. Compare migration directories between releases and review how knex_migrations is used. Done means a documented, safe rollback workflow for migrations introduced by a newer Backstage version, with tests covering the selected behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, typescript
- Domain
- backend, cli, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100