[Doc] Clarify full-compaction changelog integrality
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
Currently, the [doc](https://paimon.apache.org/docs/master/primary-key-table/changelog-producer/#full-compaction) of full compaction changelog producer states that "Full compaction changelog producer can produce complete changelog for any type of source", however, when `full-compaction.delta-commits` is greater than 1, the intermediate changes across multiple snapshots will be ignored.
Iceberg CDC refers to this as [net changes](https://iceberg.apache.org/docs/latest/spark-procedures/#change-data-capture), and Snowflake refers to this as Minimum-delta changes, both differ from a "complete" changelog. So I think this also worth clarifying in the Paimon doc, because we usually consider net changes and complete changes to be different.
### Solution
I think we should clarify that the full compaction changelog producer will only output complete changes when `full-compaction.delta-commits` is set to 1; when `full-compaction.delta-commits` is set to a value greater than 1, intermediate changes across the serveral delta snapshots will be ignored.
cc @JingsongLi What do you think?
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked Paimon documentation section for the full compaction changelog producer and read the explanation of full-compaction.delta-commits. Clarify that complete changes apply when the value is 1, while values greater than 1 ignore intermediate changes across snapshots; done when the distinction from net changes is unambiguous.
Written by the indexing model from the issue text.
Assessment
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 50/100