confluentinc / confluentinc/parallel-consumer

Memory leak in PCMetrics class

Open
#859 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
299
Forks
172
PR merge metrics
No merged PRs in 30d

Description

My team is using parallel-consumer in production, and we observed increasing memory usage over time in all of our services.
After monitoring the heap, we found that this stack trace was responsible for 96% of the heap size after 3 days of uptime:

![Image](https://github.com/user-attachments/assets/2843c5e1-3e21-4b4b-94a3-f73a8e6d167b)

![Image](https://github.com/user-attachments/assets/b26d51f8-036e-4afc-b50b-09ec07360947)

This is reproducible locally. Whenever the parallel-consumer attempts to commit its offsets, it creates two new timers and adds them to the `registeredMetrics` list in PCMetrics, even if the exact tag combination already exists.

Keeping track of the `registeredMetrics` list shows this after 10 minutes (With a 1 second commit interval):
![Image](https://github.com/user-attachments/assets/657dba79-8805-4aa7-9d63-aa353929a1ac)

Changing the list to a set seems to fix the issue, but not sure if a more sophisticated method would be preferred.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.