Create multiple best locations for each tag in the TagPartitionedLogSystem
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
If a transaction commits a large amount of data that all goes in a single shard, all the mutations in that transaction will be sent to the same tlogs.
This causes higher latencies, because only a small percentage of the tlogs is involved in processes that commit.
To avoid this problem, we can create multiple best locations for each tag, and storage servers must merge mutations from all best locations. Now, a proxy can randomly send a mutation to any of the best locations for a given tag, allowing more tlogs to be involved in these commits.
When implementing this feature, we need to make sure that a given mutation is not sent to more than one best location for any tag, so that we avoid storage servers receiving the same mutation multiple times.
Contributor guide
Research direction
Start by locating the TagPartitionedLogSystem implementation and the proxy and storage-server mutation-routing paths. Review how a tag's best location is selected and how mutations are merged, then identify tests for multiple locations and duplicate prevention. Done means mutations can use multiple best locations while each mutation reaches no more than one location per tag.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- database, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100