apache / apache/hudi

Don't keep the last commit before the earliest commit to retain

Open
#15,143 1 comment 0 reactions 1 assignee Claimed by @nsivabalan View on GitHub
component:cleaning from-jira priority:critical status:pr-available type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Don't keep the last commit before the earliest commit to retain

According to the document of {{{}hoodie.cleaner.commits.retained{}}}:

Number of commits to retain, without cleaning. This will be retained for num_of_commits * time_between_commits (scheduled). This also directly translates into how much data retention the table supports for incremental queries.
 
We only need to keep the number of commit configured through parameters {{{}hoodie.cleaner.commits.retained{}}}.
And the commit retained by clean is completed.This ensures that “This will be retained for num_of_commits * time_between_commits” in the document.

So we don't need to keep the last commit before the earliest commit to retain,If we want to keep more versions, we can increase the parameters {{hoodie.cleaner.commits.retained}}

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-3954
- Type: Improvement

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.