dmarx / dmarx/papers-feed-dev

[Epic] Schema migrations

Open
#3 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

To start with, maybe constrain attention to issues newer than https://github.com/dmarx/papers-feed/issues/1002, as it seems this is the earliest issue to use the UID scheme. Can do a separate exercise later to backport the old papers.json into issues.

* add `gh-store` label to everything with a `stored-object` label
* deduplicate
* Update paper folders to follow new UID convention (`arxiv.` prefix)
* update UID labels to follow new convention
* deduplicate
* update old style objects to use `_data` and `_meta`
* is this a comment migration? hmmm
* move `type` attribute into `_meta`
---

* UID labels should have source prefix
* should probably update paper folders to match first?
* Or maybe just flush all of that?
* need to update filenames to match the primary keys as well. bit of a PITA but whatever. not really that bad.
* new style object layout
* _data and _meta attributes
* move `type: initial_state` etc to _meta

* rebuild oldest objects in new format...
* actually, this is probably the easiest way to effect migrations.
* materialize state for old objects, then retire them and create new objects conforming to the new schema
* this feels like a kind of "deprecation", but which (potentially?) flattens/materializes the history into a new object

* would still be nice to have an operator for renaming fields
* the blocker here is largely the missing ability to remove a field (i.e. `arxivId`)
* `gh-store` operations to add:
* RENAME - for changing keys on an object
* DROP - for deleting keys on an object
* TRUNCATE - for resetting values to default without deleting a key.
* string -> empty string
* numeric -> 0
* list -> empty list
* SQUASH - flatten history into a new object, deprecate old object

---
*Copied from dmarx/papers-feed#1553*

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.