Adjust checkpoint timing adaptively according to throughput
- Dominant language
- Java
- Stars
- 2k
- Forks
- 976
- Avg merge
- 6d 15h
- Merged PRs (30d)
- 7
Description
**FEATURE REQUEST**
1. Please describe the feature you are requesting.
When using the singleentrylog mode, depending on the throughput, for example, under the throughput of 200m / s, The default value of the configuration parameter logsizelimit is 1.2g. At this time, due to the large throughput, the generation of checkpoint tasks is fast, but the execution time of checkpoint tasks is long. The speed of the two is inconsistent. A large number of tasks accumulate in the checkpoint thread, resulting in the checkpoint bits recorded in the lastmark file greatly lagging behind the real-time data. Restarting bookie will be abnormally slow. Unnecessary checkpoint tasks can be eliminated according to the execution time of checkpoint tasks. The next effective checkpoint task is generated only after one checkpoint task is executed. The specific way is to clear the accumulated tasks in the checkpoint thread pool every time a newly generated checkpoint task is added.

2. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have).
must-have
3. Provide any additional detail on your proposed use case for this feature.
Contributor guide
Assessment
This issue has not been assessed yet.