apache / apache/datafusion

Requirements for scalar UDF preimage

Open
#21,955 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

`ScalarUDFImpl::preimage` allows defining "preimage" (range of input values) for function outputs, which can be used to rewrite binary comparison expressions.

The rewrite rules for equality (`=`) or inequality (`!=`) seem intuitive. But `f(x) <= c` would also be rewritten to `x < u` where `u` is the (exclusive) upper bound. (Here is the [code](https://github.com/apache/datafusion/pull/19722/changes#diff-3f015bb6ca85b8af8daaef1f0959e0b49ec6bcbcf675b9ea2d87d52a95893773).) I'm not sure if this holds in general. Does this assume that the function must be monotonically non-decreasing?

I was trying to see if the documentation mentions monotonicity for scalar UDFs but I couldn't find it. Let me know if I misunderstood anything!

Contributor guide

Open the contributing guide

Research direction

Start with the ScalarUDFImpl::preimage entry point and the rewrite rules for equality and inequality referenced in pull request 19722. Check whether existing scalar UDF documentation addresses monotonicity or preimage requirements. Done means documenting the required assumptions and clarifying whether the <= rewrite is valid in general.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.