apache / apache/paimon

[Feature] Add lastCommittedSnapshotId commit metric and document lastScannedSnapshotId scan metric

Open
#7,588 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Motivation

Paimon's commit metrics currently provide detailed information about file counts, sizes, durations, and compaction stats, but they do not expose the snapshot ID produced by each commit. Similarly, the lastScannedSnapshotId scan metric exists in code but is missing from the metrics documentation. Adding these improves observability for operators monitoring streaming ingestion pipelines.

### Solution

Add lastCommittedSnapshotId gauge to commit metrics. Document lastScannedSnapshotId and lastCommittedSnapshotId in metrics reference

### Anything else?

_No response_

### Are you willing to submit a PR?

- [x] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Inspect the existing commit metrics implementation and the metrics reference documentation, then locate the existing lastScannedSnapshotId scan metric. Add the lastCommittedSnapshotId gauge and document both metrics; done means the commit metric exposes the produced snapshot ID and the reference covers both names.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability, stream-processing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.