Operation metadata field is not implemented for spark records
Open
area:writer
engine:spark
from-jira
priority:high
type:bug
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
If you enable the operation field using the config HoodieWriteConfig.ALLOW_OPERATION_METADATA_FIELD and you write with spark records, it will lead to data corruption (or maybe write failure, can't remember) because HoodieInternalRow does not support the field, so the indexes are incorrect when it writes the record.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8322
- Type: Bug
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.