Abstract/Refactor all transaction management logic into a set of classes
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Hudi's timeline management code sits in HoodieActiveTimeline and HoodieDefaultTimeline classes, taking action through the four stages : REQUESTED, INFLIGHT, COMPLETED, INVALID.
For sake of better readability and maintenance, we should look into reimplementing these as a state machine.
Note that this is better done after organizing the action execution classes (as in HUDI-756) in hudi-client
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-677
- Type: Improvement
- Fix version(s):
- 1.1.0
---
## Comments
08/Mar/20 09:07;yanghua;[~vinoth], [~hongdongdong] want to give a try with this issue, I will discuss with him.;;;
---
08/Mar/20 12:42;hongdongdong;yep,I will try with this issue.;;;
---
08/Mar/20 16:47;vinoth;Sounds good . This is ultra critical for the project.. So please keep me in the loop :)
;;;
---
26/Mar/20 09:41;hongdongdong;[~vinoth]
As summary said, all transaction need move out of HoodieWriteClient. A brief description of what I'm going to do: [https://docs.google.com/document/d/1-hXvcpQz42zORDlrDhJ9xEf33cmynw9hwtIb2K5DeL0/edit?usp=sharing] . Do you have any other suggestions or reminders;;;
---
26/Mar/20 20:51;vinoth;[~hongdongdong] We cannot break any existing APIs on the HoodieWriteClient, the change has to be internal to this class..
How about I write up a concrete proposal, given I have context into the various usage patterns/dependencies and you can add your thoughts on top and drive the implementation? ;;;
---
29/Mar/20 03:22;vinoth;Let me take this over.. if you don't mind [~hongdongdong] .. we can discuss on the PR.. ;;;
---
29/Mar/20 05:45;hongdongdong;[~vinoth] Of course not. ;;;
---
04/Apr/20 02:54;vinoth;After spending some time, accurately added new description..
For sake of multi engine support, I think the real issue would be to first clean up teh calling code i.e the various action executions that interact with the timeline.. I created HUDI-756 to do this for cleaning.. Will follow up with other actions as well..
We can park this one until those are done.. ;;;
---
29/Apr/20 18:40;vinoth;cc [~pwason] this is also something we need to do, to clean up tech debt.. also a good learning task.. we will have to introduce new actions for metadata compaction may be... and it will be usefiul to be conversant with this part of the code as well ;;;
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.