[Feature] Create Dela log after each commit
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar.
### Motivation
Motivation is to make paimon readable by databricks sql endpoints
### Solution
After each commit we can create delta log which will have a list of underlying parquet files
### Anything else?
Is there a way to determine parquet files when creating manifest in paimon ?
### Are you willing to submit a PR?
- [X] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
No source file or test is named; start by tracing Paimon’s commit and manifest creation paths to determine how the underlying Parquet files are identified. Clarify the required Delta log format and compatibility expectations for Databricks SQL endpoints, then define completion as producing a valid log after each commit with the committed file list.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100