apple / apple/foundationdb

Improve commit load balancing

Open
#2,600 0 comments 0 reactions 1 assignee Claimed by @etschannen View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

The current load balancing algorithm for sending commits to proxies is completely random. This leads to high tail latencies when a lot of data is all sent to the same proxy.

A better strategy would be to track the amount of bytes each client has currently outstanding to each proxy, and always send the next commit to the least loaded proxy.

The interaction with https://github.com/apple/foundationdb/pull/2520 needs to be considered. Since some proxies will have more client connections, the balancing should not be completely even.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.