Change append-distributed tables into write-anywhere tables
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
One the one hand, we want to remove the current append-distributed table feature that practically no one uses and is not MX-compatible. On the other hand, we'll at some want a table type which allows local writes from any node and append-distributed tables already meet most of the criteria, plus they are used extensively in tests. Hence, I'd like to transform the current implementation in a way that allows us to keep our test coverage and planning implementation, and maybe provide some level of backwards compatibility on the off chance that someone used it.
Some of the steps / PRs I am considering for removing current append-distributed table functions:
- [x] https://github.com/citusdata/citus/pull/5356
- [x] https://github.com/citusdata/citus/pull/5359
- [x] https://github.com/citusdata/citus/pull/5361
- [x] https://github.com/citusdata/citus/pull/5399
- [x] https://github.com/citusdata/citus/pull/5444
- [x] #5454
- [ ] Maybe change master_create_empty_shard into create_range_shard(shardminvalue text, shardmaxvalue text, nodename text, nodeport int), sync metadata
Follow up steps:
- [ ] Implement insert/copy into local shard
- [ ] Create at least one shard per node in create_distributed_table and citus_activate_node
- [ ] Decide on shard rotation policy (time-based, size-based)
- [ ] How do we move data from append-tables into hash-table? Do we use insert..select on a range of values (as usual), or move data from shards that have been rotated?
- [ ] Implement update/delete support
Contributor guide
Assessment
This issue has not been assessed yet.