ConcurrentWriteConflictResolver handle consistent hashing bucket resizing & write
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
In consistent hashing bucket index case, though we allow concurrent resizing & write, the conflict solver cannot recognize them and will choose to abort.
We should let the conflict resolver identify this case, like we did for compaction & writes.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-5644
- Type: Bug
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with ConcurrentWriteConflictResolver and trace the consistent-hashing bucket-index path, then compare it with the existing compaction-and-write handling described in HUDI-5644. The work is done when concurrent bucket resizing and writes are recognized by the resolver instead of being aborted, with regression coverage for that case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100