Add separate control for compaction operation sync/async mode
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
*Problem Review*
The compact operation sync/async in CompactionFunction is now controlled by FlinkOptions#COMPACTION_ASYNC_ENABLED
{code:java}
public CompactFunction(Configuration conf) {
this.conf = conf;
this.asyncCompaction = StreamerUtil.needsAsyncCompaction(conf);
}
{code}
While in fact it cannot be switched to sync mode because the pipeline defined by sync compaction will only include the clean but not compact operators.
{code:java}
// compaction
if (StreamerUtil.needsAsyncCompaction(conf)) {
return Pipelines.compact(conf, pipeline);
} else {
return Pipelines.clean(conf, pipeline);
}
{code}
*Improvement*
Add another separate control switch for compaction operation sync/async mode.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4329
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.