apache / apache/hudi

Eliminate the shouldLock flag for TimeGenerator(phase2)

Open
#16,257 1 comment 0 reactions 1 assignee Assigned to @danny0405 View on GitHub
area:core from-jira priority:critical type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

In HUDI-1623, we introduced the TimeGenerator, but left a shouldLock flag on the interfaces which increases the complexity for maintainance.
Basically, we need to make the lock provider as reentrant to alleviate the situration.

Several ideas here (aborted):

1. introduces the isLocked interface on the LockProvider and uses that to decide on whether we need a separate lock for the TimeGenerator (disapproved because this method has no guarantee of atomicity)

2. create temporary marker files for passing the flag (does not midiate the complexity, the caller stills needs to care about whether we should add lock or not)
3. set the shouldLock flag into the TimeGenerator instance (same as 2, does not really soves the problem)

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-6930
- Type: Improvement
- Epic: https://issues.apache.org/jira/browse/HUDI-6243
- Fix version(s):
- 1.1.0

---

## Comments

23/Sep/24 01:54;danny0405;Like described in the context, there is no good sulution now and it is not a blocker because TimeGenerator is a developer API, move it to phase2 and would rethink it in the future.;;;

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.