apache / apache/iggy

feat(go): response body buffer allocates fresh on every poll

Open
#3,452 0 comments 1 reaction 1 assignee Claimed by @matanper View on GitHub
go
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

### Description

The Go client's TCP read path allocates a new `[]byte` for the response body on every RPC. For consumers polling at high rates, this is the largest remaining contributor to allocation churn on the read side.

A previous issue (#3441) addressed the request-path allocations and the 8-byte response status header. This issue covers what remains: the variable-sized response body, which the parsed `PolledMessage` references via slices in `Message.Payload` and `Message.UserHeaders`. Because of that aliasing, the buffer must outlive deserialization, so the transport layer can't recycle it on its own.

### Affected area / component

Go SDK

### Proposed solution

Add a new method `PollMessagesInto` that reads the response body into a caller-provided buffer, growing it as needed, and returns the parsed `PolledMessage` plus the (possibly reallocated) backing slice for reuse on the next call.

```go
func (c *IggyTcpClient) PollMessagesInto(
ctx context.Context,
streamId iggcon.Identifier,
topicId iggcon.Identifier,
consumer iggcon.Consumer,
strategy iggcon.PollingStrategy,
count uint32,
autoCommit bool,
partitionId *uint32,
buf []byte,
) (*iggcon.PolledMessage, []byte, error)

Usage

var buf []byte
for {
var polled *iggcon.PolledMessage
polled, buf, err = client.PollMessagesInto(ctx, ..., buf)
if err != nil { ... }
// process polled.Messages — Payload/UserHeaders alias buf
}
// buf gets reused next iteration; zero allocations once cap is sufficient.
```

### Alternatives considered

_No response_

### Contribution

- [x] I'm willing to submit a pull request to implement this feature

### Good first issue

- [ ] I think this could be a good first issue for a new contributor

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.