[DISCUSSION] Should we treat `COMMIT_TIME_ORDERING` as a special case of `EVENT_TIME_ORDERING` ?
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
From the current code structure, we would treat the merge policy of `COMMIT_TIME_ORDERING` as a separate logic, but from a business perspective, should we treat it as a special case of `EVENT_TIME_ORDERING` where the ORDERING VALUE for each record is the same? For example, right now it's represented by `int: 0`. This way we don't need to maintain two merge policies, we default to `EVENT_TIME_ORDERING`, the same record_key merge policy is handled in the order of `transaction_time` and `event_time`. This will help us in the future to maintain the code and deal with the various merge problems we have encountered.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the existing COMMIT_TIME_ORDERING and EVENT_TIME_ORDERING merge-policy paths, including how transaction_time and event_time determine record ordering. Clarify whether the policies can share one implementation and define the required behavior before identifying affected tests and validating the change.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100