jamesplease / jamesplease/api-pls
True migration support
- Dominant language
- JavaScript
- Stars
- 10
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Right now, api-pls does not support migrations. What it does do is synchronize the database with the Resource Models defined in your file system. This issue describes a system to support true migrations.
For starters, the existing functionality needs to be renamed to `sync`. It should be documented that running sync can cause you to lose data. For instance, if during synchronization, a column is marked for removal, then all data in that column is gone. There is no way to move data from that column to another.
What if, instead, want to transform the data in that column, or move it to a new column? That requires a migration.
Here's what I'm thinking:
### Migration directory
A Migration directory contains three _types_ of files, although there may be numerous files of each type. We will use the following example in this issue:
A Resource Model has `first_name` and `last_name` attributes. The developer wishes to merge these into a new column, `full_name`, and remove the other columns. A Migration can be used to accomplish this.
##### Pre-Transform Model
These are Models that are synced before any transformation occurs. These are generally used to add new columns to the table, but are generally **not** used to remove columns. The first step of a Migration is syncing all of the Pre-Transform Models.
In this case, the migration would add the `full_name` column:
```yaml
# person.pre.yaml
extends: '../../current-model'
attributes:
full_name: 'VARCHAR(30)'
```
##### Transform.js
These are JavaScript files that export a module to transform your data. All of your transform files are run second.
Let's see how we would write a transform for our example.
```js
// person.js
module.exports = function({id, data, meta, relationships}) {
var newData = {
...data,
full_name: `${data.first_name} ${data.last_name}`
};
return {
data: newData
};
}
```
##### Post-Transform Model
This is the final step of a Migration. This is where you would drop the old columns. All Post-Transform Models are synced as the final step.
```yaml
# person.post.yaml
extends:
- '../../current-model'
- './person.pre.yaml'
attributes:
# null will delete the field that is defined on any Model that this extends from
first_name: null
last_name: null
```
My hopes and dreams is that this could all be in a transaction, but that may not be possible. If it's not, then maybe backing up the DB beforehand could be an option, with the ability to restore it if things go awry.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the existing database synchronization behavior and the proposed migration directory structure in this issue, including person.pre.yaml, person.js, and person.post.yaml. Define how sync is renamed and documented, then determine how pre-transform models, JavaScript transforms, and post-transform models execute and what completion means for the example migration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100