Add internal command to watch shard relocations progresses
- Dominant language
- Python
- Stars
- 49
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
It some cases, it's very interesting to know which shards are currently relocating and also the current progress of such operations. E.g.
- in rolling restarts as shard relocations will defer the restart
- when scaling up/down
- initial shard recovery on node start up
- ...
A common query used to see shards which are currently relocating or initializing is:
```
select node['name'], id, recovery['stage'], recovery['size'], routing_state, state, primary, table_name, relocating_node, size / 1024^3 as size_gb, partition_ident
from sys.shards
where routing_state in ('RELOCATING', 'INITIALIZING')
order by id;
```
At least for relocating shards, it should be possible to know the current progress. Not sure if this is feasible on initial recovery as the final sizes or maximum sequence numbers may not be known upfront.
Implementing an internal command which directly gives a user the current state without the need to know such system queries could help a lot.
Contributor guide
Research direction
Start with the sys.shards query in the issue to understand the current relocation and initialization state exposed by CrateDB. Define the internal command's output and determine whether progress can be reported for relocating shards and, where possible, initial recovery; done means users can inspect this state without writing the system query themselves.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- cli, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100