apache / apache/pulsar-client-go

pulsar-client-go appears to report message too large before compressing it.

Open
#447 10 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
745
Forks
389
Avg merge
3d 20h
Merged PRs (30d)
3

Description

I'm getting "message too large" messages on some large JSON documents that, when compressed, are nowhere near the limit.

in `producer_partiton.go`:
```go
// if msg is too large
if len(payload) > int(p.cnx.GetMaxMessageSize()) {
p.publishSemaphore.Release()
request.callback(nil, request.msg, errMessageTooLarge)
p.log.WithError(errMessageTooLarge).
WithField("size", len(payload)).
WithField("properties", msg.Properties).
Error()
p.metrics.PublishErrorsMsgTooLarge.Inc()
return
}
```

Correct me if I'm wrong, but this appears to check the payload size before any compression.

Whereas in `ProducerImpl.java`:
```java
int compressedSize = compressedPayload.readableBytes();
if (compressedSize > ClientCnx.getMaxMessageSize() && !this.conf.isChunkingEnabled()) {
compressedPayload.release();
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.