apache / apache/rocketmq-connect

【BUG】Connect runtime out of memory

Open Beginner friendly
#542 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
141
Forks
120
PR merge metrics
No merged PRs in 30d

Description

**BUG REPORT**

1. Please describe the issue you observed:

- What did you do (The steps to reproduce)?

- What is expected to see?
task status is RUNNING
- What did you see instead?
task status is FAILED, and reason is OutOfMemory.
2. Please tell us about your environment:

3. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
a. run `jmap -heap 16 ` found memory top 1 is org.apache.rocketmq.connect.runtime.stats.ConnectStatsService.
![image](https://github.com/user-attachments/assets/4f995ad1-c947-4fce-9c3f-6d078815c1f0)
b. use arthas watch sourceTaskTimesList and sinkTaskTimesList
![image](https://github.com/user-attachments/assets/c1c76b8f-1af4-4bfc-95c0-9f0d1214e712)
![image](https://github.com/user-attachments/assets/e1eb70e3-b7f0-4283-aea6-6e837bb3a724)

**SOLUTION:**
```
private void sampling() {
this.lockSampling.lock();
try {
this.sourceTaskTimesList.add(new CallSnapshot(System.currentTimeMillis(), sourceTaskTimesTotal()));
if (this.sourceTaskTimesList.size() > (MAX_RECORDS_OF_SAMPLING + 1)) {
this.sourceTaskTimesList.removeFirst();
}
// !!! sourceTaskTimesList is wrong !!!
// !!! should be sinkTaskTimesList !!!
this.sinkTaskTimesList.add(new CallSnapshot(System.currentTimeMillis(), sinkTaskTimesTotal()));
if (this.sourceTaskTimesList.size() > (MAX_RECORDS_OF_SAMPLING + 1)) {
this.sourceTaskTimesList.removeFirst();
}
} finally {
this.lockSampling.unlock();
}
}
```

Contributor guide

Open the contributing guide

Research direction

Start by locating ConnectStatsService and its sampling() method, then inspect how sourceTaskTimesList and sinkTaskTimesList are bounded. Verify the sampling logic against the reported OutOfMemory behavior and the code shown in the issue. Done means both task-history lists retain only the intended maximum records and the relevant runtime tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, observability
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.