scylladb / scylladb/alternator-client-cpp
Coordinator-only nodes (server-side load balancing)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Jira task: https://scylladb.atlassian.net/browse/DRIVER-678
Jira epic: https://scylladb.atlassian.net/browse/DRIVER-29
Copied from Jira epic DRIVER-29:
The original Alternator Load Balancing design document (https://docs.google.com/document/d/1twgrs6IM1B10BswMBUNqm7bwu5HCm47LOYE-Hdhuu_8/edit) suggested also the direction of server-side load balancing for unmodified clients. One of the implementations suggested was frontend (coordinator-only) nodes. Basically the idea is to have a regular Alternator node without data (owning zero tokens) serve as a load balancer to the other nodes that contain the data.
The purpose of this issue is to start testing and documenting this approach. I want to do the following:
- Write a test, perhaps in scylla.git's test/topology_custom (not here), that zero-token nodes actually work. We want to check it works for both tablets and vnodes-based tables, because currently Alternator defaults to using vnodes (but we can easily create an Alternator table with tablets for testing).
- Perform a very rudimentary performance test. The goal of this test isn't to measure absolute performance (so it won't need any nice beefy VMs) but just to measure relative performance between coordinator-only and data-holding nodes: if a data-holding Alternator nodes can finish N requests per second, we want the coordinator to be able to handle significantly more than N requests per second, otherwise it won't be useful as a load balancer (or, if we take larger nodes, this will be a very expensive solution)... In other words if coordinator is added to a K-node cluster (K=3,4,...), can we load the coordinator to 100% CPU (and leaving the cluster partially unused), or is the coordinator able to drive enough requests until the cluster is 100% used?
- Document in this repository how to set up the coordinator-only node and lessons we learned about it above. Start also thinking what we can do for high-availability (the coordinator node suddenly failing). Virtual IP address? DNS?
The relevant Scylla feature, coordinator-only nodes a.k.a. zero-token nodes or proxy nodes, was requested in https://github.com/scylladb/scylladb/issues/6527 and implemented in https://github.com/scylladb/scylladb/pull/19684.
CC @swasik
Migrated from GitHub issue: https://github.com/scylladb/alternator-load-balancing/issues/51
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked Scylla zero-token-node feature and the referenced Alternator load-balancing design document. Add a test in scylla.git test/topology_custom for zero-token nodes with tablets and vnodes, then run a relative coordinator-versus-data-node performance test. Done means recording the results, documenting setup and lessons in this repository, and considering high-availability options.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases, documentation, infrastructure, performance, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100