Multi Table Transactions
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
h3. Strawman idea:
* Introduce a notion of a "database" into Hudi's core (think of it, analogous to a database server), with its own timeline.
* We introduce a parent-child relationship between the table's timeline and the database timeline i.e an action is complete only if its completed in both timelines (similar to data <=> metadata table sync today; although we can't reuse that)
* A multi table transaction will first create the action on the database timeline, then perform actions on individual tables, then finally complete it on the database timeline.
h3. Open items:
* Need to formalize the design with considerations around isolation levels, nested queries, self joins, avoid phantom reads.
* Need to layout how we can deliver this via API and SQL (Spark for now)
* How this interplays with multi-writer scenarios and async table services.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6709
- Type: Epic
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.