apache / apache/pulsar-client-go

Discarding Corrupt Message due to apparent decompression failure

Open
#1,331 10 comments 0 reactions 1 assignee Claimed by @RobertIndie View on GitHub
Dominant language
Go
Stars
745
Forks
389
Avg merge
3d 20h
Merged PRs (30d)
3

Description

#### Expected behavior

Expect more logs to help debug what is going on with decompression

#### Actual behavior
We get the message

```
{"log":{"consumerID":95,"level":"ERROR",
"msg":"Discarding corrupted message","msgID":{"entryId":1792,"ledgerId":758404,"partition":-1},"name":"","subscription":"reader-czjug","time":"2025-02-12T10:46:13.616639169Z",
"topic":"persistent://XXXX/wav/084b74e6-f4c6-4ff8-9bff-d35370e6a77b",
"validationError":1},"stream":"stdout","timestamp":1739357173616}
```
which looks like it is coming from here
https://github.com/apache/pulsar-client-go/blob/4e71a47a4f6174f883905e740ba35f2498494ed1/pulsar/consumer_partition.go#L2172-L2182

`validationError:1` appears to be a decompression error
https://github.com/apache/pulsar-client-go/blob/4e71a47a4f6174f883905e740ba35f2498494ed1/pulsar/internal/pulsar_proto/PulsarApi.pb.go#L996

which means it is coming from here I guess?
https://github.com/apache/pulsar-client-go/blob/4e71a47a4f6174f883905e740ba35f2498494ed1/pulsar/consumer_partition.go#L1228-L1232

and given no other log lines then it suggests a cause is this
https://github.com/apache/pulsar-client-go/blob/4e71a47a4f6174f883905e740ba35f2498494ed1/pulsar/consumer_partition.go#L2148-L2151

#### Steps to reproduce

Sorry I don't have reproduction steps but maybe we could add some logging here?
https://github.com/apache/pulsar-client-go/blob/4e71a47a4f6174f883905e740ba35f2498494ed1/pulsar/consumer_partition.go#L2148-L2151

#### System configuration
**Pulsar version**: 2.11.3 and 3.3.2 (we upgraded brokers and still seeing same issue)
Pulsar Golang client: `0.14.0`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.