Enable MultiTableDeltaStreamer to update a single target table from multiple source tables.
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
It is very appropriate when MultiTableDeltaStreamer completes the business of updating table a' by table a and updating table b' by table b. Because MultiTableDeltaStreamer will start two threads, one to update table a', and the other to update table b'. If the current service scenario is that table a and table b update table c at the same time, MultiTableDeltaStreamer may encounter write conflict and cannot complete the service.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3103
- Type: New Feature
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the MultiTableDeltaStreamer entry point and review the linked HUDI-3103 request for the intended behavior. Determine how multiple source tables targeting one table should avoid write conflicts, and define tests showing that the combined updates complete successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100