cockroachdb / cockroachdb/cockroach

changefeedccl: Pulsar sink error messages include full payload and use wrong format verb

Open
#166,639 1 comment 0 reactions 0 assignees View on GitHub
branch-master C-bug O-agent T-testeng
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

## Summary

The Pulsar sink's `setError` method at `pkg/ccl/changefeedccl/sink_pulsar.go:280` formats error messages using `errors.Wrapf(err, "failed to send message %d for payload %s", id, message.Payload)`. This has two issues: (1) `%d` is used for `pulsar.MessageID`, which is an interface -- this produces Go's default representation rather than a numeric ID, and (2) `%s` is used for `message.Payload` (`[]byte`), which embeds the entire message payload verbatim in the error string. For large changefeed payloads (common in production), this creates oversized error messages that consume excessive memory and log space.

A secondary issue exists at `sink_pulsar.go:154` where `buffer.Grow()` underestimates the needed capacity by at least 2 bytes due to not accounting for JSON quotes around the topic name, causing an unnecessary extra allocation per `EmitRow` call.

## Affected code

- `pkg/ccl/changefeedccl/sink_pulsar.go:280` -- `setError` error formatting
- `pkg/ccl/changefeedccl/sink_pulsar.go:154` -- `buffer.Grow` capacity underestimate

## Reproduction

Unit test sketch:

```
// Setup: create pulsarSink with mock producer that fails SendAsync
// The mock callback should invoke the registered callback with a non-nil error
// and a message with a large payload (e.g., 10KB).
//
// Action: call EmitRow, then Flush (or another EmitRow) to surface the error
//
// Assertions:
// 1. The returned error message length is bounded (not proportional to payload size)
// 2. The MessageID portion of the error is human-readable (not Go struct dump)
```

## Suggested fix direction

1. In `setError`: Change `%d` to `%v` for `MessageID`. Either remove `message.Payload` from the error message or truncate it (e.g., first 100 bytes with `"...(truncated)"` suffix).
2. In `EmitRow` JSON format path: Move the `buffer.Grow()` call after `topicBuffer` is populated, and use `topicBuffer.Len()` instead of `len(topicName)`. Alternatively, add `+2` to `len(topicName)` to account for JSON quotes.

_This issue was found via automated deep static analysis._

Jira issue: CRDB-62045

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.