Add single-writer validation if concurrency control is not enabled
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
In case of misconfiguration or unintended deployment model, where multiple writers write to the same table without concurrency control, the writer should be able to detect that through what's on the storage, e.g., the Hudi timeline, and abort the commit. If we can enforce a pre-commit check leveraging the timeline (in single-writer case, there should not be new complete/inflight commits this writer is unaware of), an accidental multi-writer case should fail one writer and prevent the table from corrupted.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4599
- Type: Improvement
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing Hudi's writer pre-commit path and how the storage timeline exposes complete and inflight commits; the issue names no file or test. Done means a single-writer deployment can detect an unobserved commit and abort one writer before corruption occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100