Regular high CPU usage of consul ?
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 39
Description
Overview of the Issue
Hi, I am using the consul as a agent to register and discover my 4000+ nodes.
I didn't use consul cluster but only single node mode (8c16g).
And I found the regular/periodic high CPU usage of the consul node:
I can't find any useful info from consul docs.
Why does the CPU reach 100% about per 2min ? And how to solve it ?
Reproduction Steps
No special steps, I run the consul with cmd:
nohup ./consul agent -server -bootstrap -bind=127.0.0.1 -client 0.0.0.0 -ui -data-dir=./data/ >> consul.log 2>&1 &
Consul info for both Client and Server
Client info
output from client 'consul info' command here
Server info
agent:
check_monitors = 0
check_ttls = 0
checks = 9577
services = 9577
build:
prerelease =
revision = db839f18
version = 1.10.1
consul:
acl = disabled
bootstrap = true
known_datacenters = 1
leader = true
leader_addr = 127.0.0.1:8300
server = true
raft:
applied_index = 95090532
commit_index = 95090532
fsm_pending = 0
last_contact = 0
last_log_index = 95090532
last_log_term = 10192
last_snapshot_index = 95086865
last_snapshot_term = 10192
latest_configuration = [{Suffrage:Voter ID:021344bf-96b7-ff88-a6b1-5feb1110d64d Address:127.0.0.1:8300}]
latest_configuration_index = 0
num_peers = 0
protocol_version = 3
protocol_version_max = 3
protocol_version_min = 0
snapshot_version_max = 1
snapshot_version_min = 0
state = Leader
term = 10192
runtime:
arch = amd64
cpu_count = 8
goroutines = 14270
max_procs = 8
os = linux
version = go1.16.6
serf_lan:
coordinate_resets = 0
encrypted = false
event_queue = 1
event_time = 4149
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 1
members = 1
query_queue = 0
query_time = 1
serf_wan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 1
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 1
members = 1
query_queue = 0
query_time = 1
Operating system and Environment details
OS, Architecture, and any other information you can provide about the environment.
- CentOS7 x86_64
Log Fragments
Include appropriate Client or Server log fragments. If the log is longer than a few dozen lines, please include the URL to the gist of the log instead of posting it in the issue. Use -log-level=TRACE on the client and server to capture the maximum log detail.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the periodic CPU spike with the provided consul agent command and inspect the supplied consul info fields, especially the check and service counts. Collect the requested TRACE logs and profiling data to identify the recurring workload; done means explaining the cause and documenting or implementing a verified remedy.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- centos, go, linux
- Domain
- infrastructure, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100