hashicorp / hashicorp/memberlist
Question: high CPU usage for a large cluster
- Dominant language
- Go
- Stars
- 4.1k
- Forks
- 478
- Avg merge
- 15h 58m
- Merged PRs (30d)
- 3
Description
I'm evaluating a memberlist based discovery on a large cluster (~ 2500 nodes).
The problem is that the prototype continuously consumes multiple (2-3) cores on each node. However, it didn't happen for smaller installations (~100 nodes).
Is this a normal cpu usage for a large cluster or some configuration issue?
I use DefaultWANConfig and ~300 bytes metadata.
Also, an attempt to rejoin initial seeds is performed once a minute. It seems to be the only way to join cluster parts after datacenter connectivity issues long enough to mark another part nodes as failed.
Contributor guide
Research direction
Start by reproducing the reported workload with DefaultWANConfig, about 2,500 nodes, 300-byte metadata, and the once-per-minute seed rejoin behavior. Measure CPU during normal operation and after datacenter connectivity recovery, then determine whether the usage is expected or indicates a configuration or implementation problem.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100