citusdata / citusdata/citus

Change append-distributed tables into write-anywhere tables

Open
#5,358 0 comments 1 reaction 0 assignees View on GitHub
mx
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

One the one hand, we want to remove the current append-distributed table feature that practically no one uses and is not MX-compatible. On the other hand, we'll at some want a table type which allows local writes from any node and append-distributed tables already meet most of the criteria, plus they are used extensively in tests. Hence, I'd like to transform the current implementation in a way that allows us to keep our test coverage and planning implementation, and maybe provide some level of backwards compatibility on the off chance that someone used it.

Some of the steps / PRs I am considering for removing current append-distributed table functions:

- [x] https://github.com/citusdata/citus/pull/5356
- [x] https://github.com/citusdata/citus/pull/5359
- [x] https://github.com/citusdata/citus/pull/5361
- [x] https://github.com/citusdata/citus/pull/5399
- [x] https://github.com/citusdata/citus/pull/5444
- [x] #5454
- [ ] Maybe change master_create_empty_shard into create_range_shard(shardminvalue text, shardmaxvalue text, nodename text, nodeport int), sync metadata

Follow up steps:
- [ ] Implement insert/copy into local shard
- [ ] Create at least one shard per node in create_distributed_table and citus_activate_node
- [ ] Decide on shard rotation policy (time-based, size-based)
- [ ] How do we move data from append-tables into hash-table? Do we use insert..select on a range of values (as usual), or move data from shards that have been rotated?
- [ ] Implement update/delete support

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.