apache / apache/datafusion

Review the need of `make_scalar_function` for `functions`

Open
#14,835 7 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

The current code converts scalars to arrays and then back after the function calculation. However, this conversion is unnecessary and can be optimized, especially in cases involving scalars. The conversion process duplicates values in the array, which doesn't add any value.

This approach was likely implemented when Scalar was not yet introduced in arrow-rs, but that’s no longer the case. With Scalar available, the conversion to arrays is redundant.

For example, the gcd function does not require an array and can instead operate directly on an i64 value #14834

### Describe the solution you'd like

Revisit the functions that use `make_scalar_function` and identify those where it is no longer necessary. If there are functions that no longer require it, remove the usage entirely to streamline the code.

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating the functions that use make_scalar_function and read the related gcd example in issue #14834. Compare each usage with the scalar-based approach described in the issue. Done means unnecessary uses are removed while the affected functions continue to behave correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.