[FEA] Bring `filter_join_indices_jit` UDF features to parity with `transform`
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
**Is your feature request related to a problem? Please describe.**
`filter_join_indices_jit` and `transform` both evaluate a row-wise UDF over typed column inputs,
but the filtered-join path does not expose the same UDF capabilities as `transform`. In
particular, it lacks a consistent model for:
- Reporting row-level UDF evaluation errors to the caller.
- Accepting precompiled LTO IR UDFs.
- Passing caller-provided device data to the UDF.
As a result, a predicate that works with `transform` may need to be rewritten or recompiled for a
filtered join. The two paths also risk developing incompatible UDF ABIs and error semantics.
**Describe the solution you'd like**
Use the same UDF representation and execution conventions in `filter_join_indices_jit` and
`transform`:
- Accept the shared CUDA or LTO IR UDF abstraction, including explicit entry-point information.
- Accept an optional pointer to caller-owned device data and pass it to the predicate using the same
ABI convention as `transform`.
- Support fallible predicates. Aggregate per-row failures and throw `cudf::evaluation_error` after
execution, consistently with `transform`.
- Support LTO IR predicates by linking the predicate fragment with the filtered-join kernel through
RTCX/`nvJitLink`.
- Keep null-aware and non-null-aware predicate signatures consistent with `transform` where their
semantics overlap.
- Share reflection, ABI-type mapping, dispatcher generation, and cache-key logic instead of
maintaining filtered-join-specific variants.
Add coverage for CUDA and LTO IR predicates with user data enabled and disabled, successful and
fallible evaluation, null-aware and non-null-aware inputs, inner/left/full join semantics, and cold
and warm JIT caches.
**Describe alternatives you've considered**
- Keep a separate filtered-join UDF contract. This limits functionality and requires callers to
maintain different versions of equivalent row-wise operations.
- Express every predicate as an AST. AST predicates are useful, but they cannot replace arbitrary
precompiled or user-defined device functions.
**Additional context**
This proposal is implemented in a prototype #23565
Contributor guide
Research direction
Start with prototype #23565, then compare filter_join_indices_jit with transform to identify their UDF representations, ABI conventions, reflection, dispatch, and cache-key paths. Review the CUDA and LTO IR behavior through RTCX/nvJitLink. Done means parity coverage for user data, successful and fallible predicates, null-aware inputs, join semantics, and cold or warm JIT caches.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100