Support using event time for timeline
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Now the instant time in timeline is the proccessing time of a record in hudi. But in some cases, users may need the event time for reading. For example, we import a mysql table to hudi realtimely, but some batch job need to calculate the data for yesterday, they need a snapshot of the mysql table at 00:00, this should be the event time, not instant time, as it may delay due to slow running or job restarting and so on. So We proposal to support user define using event time as the instant time in timeline.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-4622
- Type: New Feature
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the issue description and the linked JIRA ticket HUDI-4622; no source files, tests, or entry points are identified. Trace how Hudi represents processing time in its timeline and determine the required configuration and event-time semantics. Done should include a defined behavior for event-time snapshots and corresponding validation, but the issue does not specify where tests belong.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100