apache / apache/fluss

Optimize client bucket-level metrics

Open
#3,826 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Motivation

Currently, Fluss exposes client-side bucket-level metrics during Flink job execution. For jobs with many buckets or large-scale deployments, these high-cardinality metrics consume a significant amount of metrics storage and lead to high observability costs.

Need to reduce the default metrics cardinality and storage cost, while still allowing users to expose bucket-level metrics when they need fine-grained observability or troubleshooting.

### Solution

Phase 1: Remove the default exposure of client-side bucket-level metrics to reduce metrics storage pressure for existing Flink jobs.

Phase 2: Add a configuration option to let users enable bucket-level metrics on demand. The option should be disabled by default.

### Anything else?

_No response_

### Willingness to contribute

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing where client-side bucket-level metrics are exposed during Flink job execution and how client configuration is defined and read. Done means bucket-level metrics are absent by default, while a documented opt-in configuration exposes them for troubleshooting without changing other client metrics.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.