apache / apache/datafusion

Enable lazy argument evaluation for arbitrary scalar functions

Open
#17,992 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

If a scalar function currently want to use lazy argument evaluation it needs to implement `simplify` and rewrite itself into some other `PhysicalExpr` that implements lazy evaluation. `coalesce` being rewritten to `case` is a good example of this.

While this is fine for some situations, it would be more flexible if `ScalarUDF` implementations could control argument evaluation themselves rather than having to rewrite themselves.

There is a hint of support for this idea already in the form of `ScalarUDF#short_circuits`.

### Describe the solution you'd like

Provide some way to pass arguments whose evaluation is deferred to scalar UDF implementations so that those can control if and when arguments are evaluated.

### Describe alternatives you've considered

_No response_

### Additional context

Relates to #17982 and #17983

Contributor guide

Open the contributing guide

Research direction

Start by reading the ScalarUDF interface and its existing short_circuits support, then review the related issues #17982 and #17983 for context. Determine how scalar UDFs could receive deferred arguments and define the evaluation contract; the work is done when arbitrary scalar functions can control whether and when those arguments are evaluated.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.