apache / apache/pulsar-client-go

Prometheus metric on publisher message compression ratio

Open
#1,359 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
745
Forks
389
Avg merge
3d 20h
Merged PRs (30d)
3

Description

Is your feature request related to a problem? Please describe.
In our organisation we sink Pulsar messages for some topics directly to S3. It is useful for us to know the compression ratio that we are achieving on our payload size so that we are maximising efficiency of data storage, as we are able to grow/shrink the payload size to optimise for this.

Describe the solution you'd like
A prometheus histogram or gauge metric that producers publish on each message compression.

Describe alternatives you've considered
We considered calculating this metric ourselves, however the compression code is in the internal directory in the project. I do not think there are any public methods that would give us this data.

Additional context
N/A

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points; begin by locating the publisher's internal message-compression path and existing Prometheus metric definitions. Confirm how compressed and original payload sizes are available, then verify that a per-message compression-ratio histogram or gauge is exposed and covered by appropriate tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, prometheus
Domain
backend, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.