filecoin-project / filecoin-project/lily
Lily should support migrations for Timescale cluster nodes
- Dominant language
- Go
- Stars
- 52
- Forks
- 47
- PR merge metrics
- No merged PRs in 30d
Description
### Description
Currently, Lily migrations expect to be deployed to a single Timescale instance. We are currently exploring using lily data in a cluster environment. The cluster environment requires different migrations from their single node case. While I think Lily's migration must support multiple database targets (vanilla postgres + timescale cluster, in addition to the single node setup), the scope of this story is specific to supporting timescale cluster migrations only.
(It's worth noting that I think adding a separate branch of support for vanilla postgres might benefit by whatever pattern gets used here, even if we don't pursue vanilla postgres support today.)
### Acceptance criteria
- I can pass a bool arg or adjust a config value (or control in some other way via CLI) for `lily migrate` to enable the special cluster migration case without affecting the single node deployment case (which should be the default behavior when the new bool is not specified).
### Where to begin
The schema migrations primarily change in the two following ways:
- `create_hypertable` becomes `create_distributed_hypertable` with one extra argument which describes the columnar indicies. (the argument is a (set of?) strings)
- `create type * as enum` and `create function` both require the same statement (as-is) to be executed via `call distributed_exec` which distributes the command to the cluster.
An example of the full schema creation SQL (including the schema creation, which is not currently a concern of lily migrations) used in the cluster environment can be found at https://gist.github.com/placer14/c044e7e8c3aa4933a847ed3c29b59c46.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.