apache / apache/paimon

[Feature] The maximum watermark value of the current commit should be reported by `CommitterMetrics`

Open
#2,742 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.

### Motivation

I write paimon through the flink. After commit, I hope to report the watermark as an indicator, so that I can know the freshness of the current paimon table instead of obtaining it through the system table. That query is too slow. Through flink Once the metric report comes out, it will be easier for me to combine it with a third-party monitoring system.

### Solution

Report the watermark when committing

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating CommitterMetrics and the commit path used by Flink. Trace how the current commit watermark is available, then determine how the maximum value should be exposed as a metric and add coverage showing that monitoring consumers receive it.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, stream-processing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.