Flink Sink V2: Add PostCommitHook plugin interface
- Dominant language
- Java
- Stars
- 9.2k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 132
Description
### Feature Request / Improvement
Adds a `PostCommitHook` plugin interface to `IcebergSink` that is invoked after each successful Iceberg commit. This enables use cases like updating table properties, triggering notifications, recording metrics, or synchronizing external metadata stores.
### Query Engine
Flink
### Motivation
After a successful Iceberg commit, users often need to perform additional operations (e.g., write watermark metadata to table properties, notify downstream systems, update external catalogs). Currently there is no extension point -- users must subclass `IcebergCommitter` and override `commit()`, which requires forwarding many constructor arguments through the subclass chain.
This adds a `PostCommitHook` that is called after `operation.commit()` with the snapshot ID and snapshot summary. The hook runs in the committer's context and has access to all commit metadata.
### Changes
- New: `PostCommitHook.java` -- `@FunctionalInterface` with a single method: `void afterCommit(long snapshotId, Map summary)`
- Modified: `IcebergCommitter` -- accepts optional hook, invokes it after successful commit
- Modified: `IcebergSink.Builder` -- new `postCommitHook()` method, passed through to committer
### Compatibility
- No behavioral change when the hook is not set (null default)
- No changes to public API signatures of existing methods
- Fully backward compatible
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
Contributor guide
Research direction
Start with the new PostCommitHook.java interface, then trace IcebergCommitter's successful operation.commit() flow and IcebergSink.Builder's option handling. Add the hook plumbing described in the issue, and verify that it receives the snapshot ID and summary after commits while behavior remains unchanged when unset.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100