HoodieIndexUtils.filterKeysFromFile hardcodes AVRO fallback, ignoring configured record merger type
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Describe the problem
In [`HoodieIndexUtils.filterKeysFromFile`](https://github.com/apache/hudi/blob/master/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java#L253) the record type used to open the base file reader for key filtering is computed as:
```java
HoodieRecord.HoodieRecordType recordType =
HoodieFileFormat.fromFileExtension(FSUtils.getFileExtension(filePath.toString()))
.resolveRecordType(HoodieRecord.HoodieRecordType.AVRO);
```
`resolveRecordType` only overrides the fallback when the file format **requires** a specific record type (currently only Lance). For all other formats (Parquet, ORC, HFile) the AVRO fallback is hardcoded and ignores the merger's configured record type. This predates the Lance work (#18375).
### Impact
If a user configures a SPARK-type merger for a Parquet table, key filtering during index lookup will still open an AVRO reader, incurring unnecessary Avro<->InternalRow conversions in the key-filter code path.
This is not a correctness bug — `filterRowKeys` only reads record keys — but it is an efficiency/consistency gap flagged during review of #18375 (cf. [review thread](https://github.com/apache/hudi/pull/18375#discussion_r3036297458)).
### Proposed fix
Thread the configured merger's record type into `filterKeysFromFile` (or read it from config via an argument) and pass it as the fallback to `resolveRecordType` instead of hardcoding `AVRO`.
### Context
- Raised by @rahil-c in https://github.com/apache/hudi/pull/18375#discussion_r3069697589
- File: `hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java` around line 254 (the `// TODO: AVRO fallback` comment).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java at filterKeysFromFile and inspect its callers and the review context from #18375. Trace how the configured merger record type is available, then verify that it is used as the fallback while formats requiring a specific type, such as Lance, still override it. Done means key filtering no longer hardcodes AVRO for configurable formats.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100