apache / apache/paimon

[Docs] Explain how flink checkpoint works with paimon.

Open
#1,146 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 11h
Merged PRs (30d)
396

Description

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.

### Motivation

Paimon have two stage to complete write and commit. While a checkpoint comes, we do something in prepareSnapshot, pass the commitable to the commit stage task. In snapshotStates phase, we write these message into a state. When acknowledge checkpoint, we call the commit to generate a new paimon snapshot.
We need a docs to explain this process, explain how paimon works with flink.

### Solution

_No response_

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the checkpoint flow described in prepareSnapshot, snapshotStates, and the commit stage. Document how Paimon works with Flink, including how committable messages move through checkpoint state and how acknowledging a checkpoint generates a new Paimon snapshot. Done means the two-stage write and commit process is explained clearly for readers.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.