apache / apache/hudi

test(spark): legacy standalone incremental relations (IncrementalRelationV1/V2) have three untested branches

Open
#19,443 0 comments 0 reactions 1 assignee Claimed by @voonhous View on GitHub
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Follow-up from the review of #19404. The standalone incremental relations `IncrementalRelationV1` / `IncrementalRelationV2` (reachable only via the legacy, file-group-reader-disabled streaming path through `HoodieStreamSourceV1/V2.getBatch`) carry three branches that no test in the repo reaches. The cheapest vehicle for all three is `TestLegacyParquetReadPath.testCowIncrementalReadEqualsFileGroupReader`, which already constructs both relations directly (lines 317-319).

- [ ] Fallback/full-table-scan block (`IncrementalRelationV1.scala:235-272`, `IncrementalRelationV2.scala:225-259`, including the parallelized file-existence probe from #10480): set `hoodie.datasource.read.incr.fallback.fulltablescan.enable=true` after aggressive cleaning and assert both relations still equal the file-group-reader result. This block also carries the fix history of HUDI-2711, HUDI-3189 and HUDI-7003.
- [ ] File-path copy of `hoodie.datasource.read.incr.path.glob` (`IncrementalRelationV1.scala:196-207`, `IncrementalRelationV2.scala:181-192`): a second, independently written implementation of the option over `regularFileIdToFullPath` / `metaBootstrapFileIdToFullPath`, including a meta-bootstrap arm the MOR relations have no equivalent of. No test sets the glob on this path (#19404 covers only the file-slice copy in `MergeOnReadIncrementalRelationV1/V2`).
- [ ] `hoodie.datasource.read.incr.schema.use.end.instanttime` (`IncrementalRelationV1.scala:121-129`, `IncrementalRelationV2.scala:106-114`, introduced by HUDI-1301 / #2125): set by no test anywhere, so the `useEndInstantSchema=true` branch is uncovered.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.