Store current Instant time to checkpoint in Hudi Flink Sink
Open
engine:flink
priority:low
type:improvement
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
In our org, we need to switch from Flink hudi ingestion to Spark hudi ingestion in the case of disaster recovery. In this case, we need to find our what's the corresponding kafka offset of the last committed instant. Flink state API can fetch the offset info from checkpoint, but we need a flag to see whether it matches to particular Instant. Thus, we would like to add the current instant into the checkpoint.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8903
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.