hashicorp / hashicorp/consul

Constant cpu usage

Open
#17,996 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

#### Overview of the Issue

Consul is doing nothing. Consul server is doing nothing. No one is connected. Still, the cpu usage of consul us around 2% constantly all the time.

#### Reproduction Steps

Install consul. Start consul.

### Consul info for both Client and Server

```
$ consul --version
Consul v1.15.0
Revision 53f65dc3
Build Date 2023-02-24T01:39:35Z
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)
$ consul info
agent:
check_monitors = 0
check_ttls = 1
checks = 138
services = 137
build:
prerelease =
revision = 7ce982ce
version = 1.15.3
version_metadata =
consul:
acl = disabled
known_servers = 4
server = false
runtime:
arch = amd64
cpu_count = 64
goroutines = 199
max_procs = 64
os = linux
version = go1.20.4
serf_lan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 170
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 126986
members = 32
query_queue = 0
query_time = 1
```

### Operating system and Environment details

Fedora 29

### Log Fragments

$ top -b -n 10 -d 1 -p $(pgrep consul)

```
top - 04:05:33 up 39 days, 11:08, 3 users, load average: 0.79, 0.72, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.4 us, 0.3 sy, 0.1 ni, 99.1 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 128876.3 total, 28613.5 free, 9392.0 used, 90870.8 buff/cache
MiB Swap: 8192.0 total, 7678.0 free, 514.0 used. 119982.1 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819380 89440 60492 S 0.0 0.1 0:06.79 consul

top - 04:05:34 up 39 days, 11:08, 3 users, load average: 0.79, 0.72, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.3 us, 0.2 sy, 0.0 ni, 99.4 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 128876.3 total, 28613.5 free, 9392.0 used, 90870.8 buff/cache
MiB Swap: 8192.0 total, 7678.0 free, 514.0 used. 119982.1 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819380 89440 60492 S 2.0 0.1 0:06.81 consul

top - 04:05:35 up 39 days, 11:08, 3 users, load average: 0.79, 0.72, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.1 ni, 99.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 128876.3 total, 28613.5 free, 9392.0 used, 90870.8 buff/cache
MiB Swap: 8192.0 total, 7678.0 free, 514.0 used. 119982.1 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819380 89440 60492 S 2.0 0.1 0:06.83 consul

top - 04:05:36 up 39 days, 11:08, 3 users, load average: 0.79, 0.72, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.3 us, 0.1 sy, 0.0 ni, 99.5 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 128876.3 total, 28611.6 free, 9393.8 used, 90870.9 buff/cache
MiB Swap: 8192.0 total, 7678.1 free, 513.9 used. 119980.2 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819380 89440 60492 S 3.0 0.1 0:06.86 consul

top - 04:05:37 up 39 days, 11:08, 3 users, load average: 0.79, 0.72, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.1 sy, 0.1 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 128876.3 total, 28611.6 free, 9393.8 used, 90870.9 buff/cache
MiB Swap: 8192.0 total, 7678.1 free, 513.9 used. 119980.2 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819380 89440 60492 S 1.0 0.1 0:06.87 consul

top - 04:05:38 up 39 days, 11:08, 3 users, load average: 0.73, 0.71, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.2 us, 0.1 sy, 0.1 ni, 99.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 128876.3 total, 28611.6 free, 9393.8 used, 90870.9 buff/cache
MiB Swap: 8192.0 total, 7678.1 free, 513.9 used. 119980.2 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819380 90200 60492 S 3.0 0.1 0:06.90 consul

top - 04:05:39 up 39 days, 11:08, 3 users, load average: 0.73, 0.71, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 128876.3 total, 28611.8 free, 9393.6 used, 90870.9 buff/cache
MiB Swap: 8192.0 total, 7678.1 free, 513.9 used. 119980.4 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819380 91784 60492 S 1.0 0.1 0:06.91 consul

top - 04:05:40 up 39 days, 11:08, 3 users, load average: 0.73, 0.71, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.2 us, 0.1 sy, 0.2 ni, 99.4 id, 0.0 wa, 0.0 hi, 0.1 si, 0.0 st
MiB Mem : 128876.3 total, 28611.8 free, 9393.6 used, 90870.9 buff/cache
MiB Swap: 8192.0 total, 7678.1 free, 513.9 used. 119980.4 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819636 93092 60492 S 14.0 0.1 0:07.05 consul

top - 04:05:41 up 39 days, 11:08, 3 users, load average: 0.73, 0.71, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.1 us, 0.1 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 128876.3 total, 28611.8 free, 9393.6 used, 90870.9 buff/cache
MiB Swap: 8192.0 total, 7678.1 free, 513.9 used. 119980.4 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819636 93092 60492 S 1.0 0.1 0:07.06 consul

top - 04:05:42 up 39 days, 11:08, 3 users, load average: 0.73, 0.71, 0.32
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.0 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 128876.3 total, 28609.8 free, 9395.5 used, 90870.9 buff/cache
MiB Swap: 8192.0 total, 7678.1 free, 513.9 used. 119978.4 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
22428 sysavtb+ 30 10 819636 93092 60492 S 2.0 0.1 0:07.08 consul
```

Nothing is printed to logs. The machine has E5-4650 2.7GH with 64 cores.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the report with Consul v1.15.3 using the stated install-and-start steps, then compare `consul info` with repeated `top` output to isolate the idle CPU activity. Done means identifying the cause of the unexplained usage and documenting or correcting the behavior with evidence from a comparable idle run.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.