Avoid searching for "_hoodie_operation" field in data files
Open
from-jira
priority:high
type:devtask
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
When Hudi tries to add metadata fields to the table schema, it tries to check if "_hoodie_operation" field exists by searching all data file headers. This search could take long to finish. Therefore, we should try to avoid checking this field as much as possible.
The potential solution is to add a table property to indicate if this field exist. If not, no need to search.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-7121
- Type: Task
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.