apache / apache/druid

[Bug] Memory Leak in ConcurrentGrouper due to missing ThreadLocal.remove() cleanup

Open Beginner friendly
#18,942 0 comments 0 reactions 0 assignees View on GitHub
Uncategorized problem report
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version
Current Master / Trunk (Identified in org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.java)

### Description
Problem Summary:
A potential memory leak exists in ConcurrentGrouper. The class uses a ThreadLocal to manage thread-specific aggregation buffers but fails to clean it up in the close() method.

Detailed Analysis:
Initialization: A ThreadLocal field named threadLocalGrouper is initialized in the constructor (Line 184) to bind SpillingGrouper instances to processing threads.

The Bug: The close() method (Line 479) iterates through the groupers list to close them, but misses calling threadLocalGrouper.remove().

Impact: In Druid's thread-pool environment (Historical/Broker nodes), the processing threads remain alive after the query finishes. Because .remove() is not called, the SpillingGrouper instances (and their underlying ByteBuffers) remain referenced by the threads' ThreadLocalMap, preventing Garbage Collection. This leads to a gradual Memory Leak.

Steps to reproduce the problem:
1. Inspect org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.java.

2. Observe that threadLocalGrouper is created via ThreadLocal.withInitial(...).

3. Check the close() implementation; it lacks the necessary cleanup call.

Relevant Code Snippet:
// Location: org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.java

@Override
public void close()
{
if (!closed) {
closed = true;
groupers.forEach(Grouper::close);
// MISSING: threadLocalGrouper.remove(); <--- LEAK SOURCE
}
}

Proposed Fix:
Add threadLocalGrouper.remove() inside the close() method to ensure resources are detached from the thread.

Contributor guide

Open the contributing guide

Research direction

Read org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.java, focusing on the threadLocalGrouper field and the close() implementation. Verify that closing the grouper detaches the thread-local resource and preserves the existing groupers cleanup behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Bug
Difficulty
1/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.