[FEA] Expose filter_join_indices in Java
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
Java callers can reuse an equality hash table across probe tables with `HashJoin`, but cannot compose the resulting gather maps with an AST predicate. As a result, callers using `Table.mixedInnerJoinGatherMaps` or `Table.mixedLeftJoinGatherMaps` rebuild the hash table on every call.
libcudf already implements the desired composition: use `cudf::hash_join::{inner,left}_join`, then pass the resulting indices to `cudf::filter_join_indices`. The mixed inner/left/full implementation was rewritten this way in #23012, but `filter_join_indices` has no Java/JNI binding.
**Describe the solution you'd like**
Add a Java/JNI binding for `cudf::filter_join_indices` that accepts:
- left and right input `GatherMap`s;
- left and right tables containing the columns used by the predicate;
- a `CompiledExpression`;
- the join kind, supporting at least inner and left joins.
It should return newly allocated left and right `GatherMap`s without modifying or taking ownership of the input maps. The JNI implementation can borrow the gather-map addresses, wrap them in `cudf::device_span`, and call the existing libcudf API.
This would allow callers to compose the existing reusable-hash APIs with filtering:
```java
try (HashJoin rightHash = new HashJoin(rightKeys, compareNullsEqual)) {
GatherMap[] equalityMaps = leftKeys.innerJoinGatherMaps(rightHash);
GatherMap[] result = Table.filterJoinGatherMaps(
equalityMaps[0], equalityMaps[1],
leftConditional, rightConditional,
condition, JoinKind.INNER);
}
```
Tests should cover:
- reusing one `HashJoin` across multiple probe tables;
- equivalence with `mixedInnerJoinGatherMaps` and `mixedLeftJoinGatherMaps`;
- duplicate equality keys and a left row whose conditional matches all fail;
- nullable predicates;
- empty and mismatched gather maps.
**Describe alternatives you've considered**
Adding `HashJoin` overloads to the existing `mixed*JoinGatherMaps` methods would solve the immediate hash-table rebuild, but would preserve the monolithic mixed-join API and prevent the filter from being composed with other join implementations such as sort-merge join.
Spark RAPIDS JNI provides a similar custom primitive, `JoinPrimitives.filterGatherMapsByAST`, but cuDF Java should bind the existing libcudf primitive instead of duplicating its native implementation.
**Additional context**
This addresses hash-table reconstruction and allocator churn. It does not eliminate the materialized equality-join gather maps introduced by the join-then-filter composition; reducing that peak memory usage would be separate work.
The libcudf API is already present in 26.08 and current main. A Java/JNI binding in 26.10 would unblock callers upgrading from 26.06 that need reusable hash joins for mixed inner and left joins.
Related: #22124, #23012.
Contributor guide
Research direction
Start by tracing the existing Java/JNI entry points for Table.mixedInnerJoinGatherMaps, Table.mixedLeftJoinGatherMaps, HashJoin, and the libcudf filter_join_indices API. Add coverage for reusable HashJoin instances, equivalent mixed joins, duplicate keys, nullable predicates, and empty or mismatched gather maps. Done means Java callers receive new filtered gather maps without modifying or owning the inputs, with inner and left joins supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, java
- Domain
- api, data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100