tikv / tikv/pd

PD OOM and `transport.(*serverHandlerTransport).HandleStreams.func5` continuously occupying memory.

Open
#9,649 1 comment 0 reactions 0 assignees View on GitHub
needs-type
Dominant language
Go
Stars
1.2k
Forks
783
Avg merge
5d 21h
Merged PRs (30d)
36

Description

PD OOM after upgrade to 8.5.2 and rolling restart

Image

![Image](https://github.com/user-attachments/assets/6d2c9d79-5729-4ecf-9fcd-80c6b86e712b)

The root is 11G and 22G. And diff is from `transport.(*serverHandlerTransport).HandleStreams.func5`

![Image](https://github.com/user-attachments/assets/dc763326-e0b4-4867-9678-85dc9a5919aa)

![Image](https://github.com/user-attachments/assets/f4d28db7-c2d3-49ac-8e39-eb83668ba61b)

And the number of goroutine is stable

Image

Grpc query is also stable

Image

Http query is little

Image

We do nothing and the memory drops after the Heartbeat latency decrease.

Image

Image

https://github.com/tikv/pd/issues/8882 seem related to this issue?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.