apache / apache/pulsar-dotpulsar

Memory problem

Open
#245 5 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
271
Forks
87
PR merge metrics
No merged PRs in 30d

Description

### Description

Hi Team, I encountered a problem when sending a large message (130M). During the sending process, the memory usage rose to 400M and was not reclaimed after the method ended. Even after I manually triggered a garbage collection, the memory did not drop back to within 20M. When receiving the message, the memory suddenly spiked to 1G. After I manually triggered a garbage collection, it dropped back to 900M.

### Reproduction Steps

1.Send a big message(170M):
byte[] byteArray = null;
using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
byteArray = new byte[fs.Length];
fs.Read(byteArray, 0, byteArray.Length);
}
var messageIdT = await producer.NewMessage().Send(byteArray);
2.The memory will increase by 390M after Send()
3.Receive message
await foreach (var message in consumer.Messages())
{
await consumer.Acknowledge(message);
}
4.When this message is received, the memory usage will start to rise, reaching one gigabyte, and after completing this cycle, the memory is not reclaimed.

### Expected behavior

Want memory to be reclaimed

### Actual behavior

I observed that when DotPulsar sends large messages, it takes some time. During this period, the memory usage starts to rise, and even after the sending is completed, the memory is not reclaimed. When receiving messages, the memory usage also rises sharply. And because it's in a foreach loop, I'm not sure if I can manually reclaim the memory.

### Regression?

_No response_

### Known Workarounds

_No response_

### Configuration

1. .net Framework 4.6.2
2. DotPulsar 3.6.0
3. Windows 10
4. Any CPU

### Other information

I'm sorry. I don't know

Contributor guide

Open the contributing guide

Research direction

Reproduce the report with the 170M byte array and DotPulsar 3.6.0, starting at producer.NewMessage().Send and the consumer.Messages()/Acknowledge loop. Trace memory ownership through sending and receiving; done means the large message no longer remains unexpectedly retained and a regression test covers the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.