Revisit ENUM handling in Spark parquet reader and writer
Open
from-jira
priority:blocker
type:devtask
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Spark Avro reader logic treats Avro ENUM type as String when converted to Spark StructType schema, which is converted back to Avro String type if it goes through Row to GenericRecord conversion. This is different from our previous logic where the record is directly read out as Avro GenericRecord.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-8660
- Type: Sub-task
- Parent: https://issues.apache.org/jira/browse/HUDI-9108
- Fix version(s):
- 1.1.0
---
## Comments
12/May/25 19:44;linliu;Where does it translate ROW to GenericRecord?
Can we use the original AVRO schema? If not, why?;;;
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.