apache / apache/incubator-xtable
Add Support for delta generated columns when creating InternalSchema from a delta source table
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 212
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 16
Description
### Feature Request / Improvement
Currently, delta generated columns gets filter out when creating InternalSchema: https://github.com/apache/incubator-xtable/blob/main/xtable-core/src/main/java/org/apache/xtable/delta/DeltaSchemaExtractor.java#L218
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in xtable-core/src/main/java/org/apache/xtable/delta/DeltaSchemaExtractor.java at the filtering logic around line 218. Trace how an InternalSchema is created from a Delta source table and determine how generated columns are currently excluded. Done means generated columns from the Delta schema are retained in the resulting InternalSchema.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100