apache / apache/pulsar-client-cpp

[Bug] Keep-alive timeout when send/receive large message

Open
#425 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
71
Forks
90
Avg merge
2h 33m
Merged PRs (30d)
3

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/pulsar-client-cpp/issues) and found nothing similar.

### Version

client 3.2
server 3.2.2

### Minimal reproduce step

Use a producer to send one 300M message to the broker using synchronous mode with 5M bandwidth, just make sure the message to be sent in almost one minute.
BTW, it works fine for small message, for example, messages less than 100M can be successfully sent or consumed within 5M bandwidth

### What did you expect to see?

Send successfully

### What did you see instead?

2024-04-28 16:58:11.157 WARN [40520] D:\a\pulsar-client-cpp\pulsar-client-cpp\lib\ClientConnection:1158 | [10.214.4.3:49457 -> 8.218.192.194:6650] Forcing connection to close after keep-alive timeout
2024-04-28 16:58:11.157 ERROR [40520] D:\a\pulsar-client-cpp\pulsar-client-cpp\lib\ClientConnection:1234 | [10.214.4.3:49457 -> 8.218.192.194:6650] Connection closed with ConnectError

### Anything else?

When I use asynchronous mode in the producer side, this issue is mitigated, while the consumer stuck into it. I find the keepalive interval is hardcoded in ClientConnection.cc and the value is 30 seconds, and I also find it executed by a deadline timer which run in a separated thread, thus I am not sure if this has something to do with the large message sending or receiving

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start in ClientConnection.cc, especially the hardcoded 30-second keep-alive interval and its deadline timer. Reproduce with client 3.2 and server 3.2.2 using a synchronous 300M message at 5M bandwidth, then compare synchronous and asynchronous producer behavior and consumer behavior. Done means the large send or receive completes without the keep-alive timeout closing the connection.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.