Refactor TableSchemaResolver to remove `includeMetadataFields` flags
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 112
Description
Currently, `TableSchemaResolver` to pass `includeMetadataFields` in its APIs that would selectively remove metadata fields from the returned schemas. There are multiple issues with this flag:
# It's applied inconsistently: sometimes it just means that meta fields {_}won't be added{_}, and sometimes it means fields _would be removed_ even if present
# This flag actually spells the usage context into TableSchemaResolver: it's highly contextual whether caller wants to remove, omit such meta-fields, or take the schema as is and should not be spilled into the Resolver itself.
# B/c of it there's no way to know actually what was actual schema the data was written with (b/c flag might not only omit, but also change the original schema)
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-3626
- Type: Bug
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.