[Umbrella] Suppors LakeSoul integration
- Dominant language
- Scala
- Stars
- 2.4k
- Forks
- 1k
- PR merge metrics
- No merged PRs in 30d
Description
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues) and found no similar issues.
### Describe the proposal
[LakeSoul](https://github.com/meta-soul/LakeSoul) implements incremental upserts for both row and column and allows concurrent updates. LakeSoul uses LSM-Tree like structure to support updates on hash partitioning table with primary key, and achieve very high write throughput (30MB/s/core) on cloud object store like S3 while providing optimized merge on read performance. LakeSoul scales metadata management and achieves ACID control by using PostgreSQL. LakeSoul provides tools to ingest CDC and log streams automatically in a zero-ETL style. Kyuubi could support the integration with LakeSoul.
### Task list
Suppors LakeSoul integration.
### Are you willing to submit PR?
- [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
- [X] No. I cannot submit a PR at this time.
Contributor guide
Research direction
Start with the linked LakeSoul project and review Kyuubi's existing integration entry points; this issue names no files or tests. Clarify the supported integration scope and requirements first, then define validation for a working, tested LakeSoul integration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql
- Domain
- data-engineering, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100