Add more states to rebalancer progress
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
Right now the `get_rebalance_progress()` only shows 3 states encoded as numbers:
0: waiting
1: moving
2: moved
This is useful, but the problem is that a lot is happening during moving. The 3 main stages are:
1. Copying the data
2. Creating the indexes
3. Waiting for other colocated shards to finish
It would be very useful to indicate these different states to the user. The main reason it would be useful is that if the size of the shard does not grow anymore, then a user might think the rebalance is stuck. However, it is only expected that the size of the data grows when copying the data. In the other two cases, no change in data size is expected.
A few things to keep in mind when fixing this:
1. How do we add new states? e.g. does it cause problems if we add new numbers for existing users of the `get_rebalance_progress` function? If so, maybe we can add a new column that contains an enum. This would also be a bit more readable, since you would not have to know the mapping.
2. Currently all state changes are done directly from the `rebalance_table_shards()` call. However these new states would need to be set in the `citus_move_shard_placement()` call (which `rebalance_table_shards` calls internally in a different transaction).
Originally suggested on Reddit: https://www.reddit.com/r/PostgreSQL/comments/ph8zvk/shard_rebalancing_in_the_citus_101_extension_to/hbhpfwe/?context=3
Contributor guide
Assessment
This issue has not been assessed yet.