hashicorp / hashicorp/consul

Memory Leak With consul client

Open
#12,564 4 comments 1 reaction 0 assignees View on GitHub
theme/performance type/bug
Dominant language
Go
Stars
30.1k
Forks
4.6k
Avg merge
1d 18h
Merged PRs (30d)
39

Description

#### Overview of the Issue

We use Consul with Haproxy and the consul client sometimes eats all the memory in the server and gets killed due to Out Of memory.
![image](https://user-images.githubusercontent.com/16200452/158520141-c9359f51-688b-4120-a22d-9c619b0e133f.png)

![image](https://user-images.githubusercontent.com/16200452/158531972-324dd75b-154d-43fe-a760-2ba1709551bc.png)

#### Reproduction Steps

It usually happens when we restart the consul client.

### Consul info for both Client and Server

Client info

We use the latest consul client (1.11.4)

Server info
Consul v1.10.2
Revision 3cb6eeedb
Protocol 2 spoken by default, understands 2 to 3 (agent will automatically use protocol >2 when speaking to compatible agents)

### Operating system and Environment details

We use Centos 7

### Log Fragments

[consul-debug-2022-03-16T11-15-39+0700.tar.gz](https://github.com/hashicorp/consul/files/8259017/consul-debug-2022-03-16T11-15-39%2B0700.tar.gz)

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the attached Consul debug archive and reproducing the issue while restarting the Consul client. Trace the memory growth and verify that repeated restarts no longer cause unbounded memory use or an out-of-memory kill.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.