apache / apache/datafusion

feat: Update string UDF's to have return type == input type where appropriate

Open
#20,585 11 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

Quite a few UDF's currently only return Utf8 or LargeUtf8 even if the input type is Utf8View. Ideally, all functions that consume strings and return the same will return values in the same data type as the primary (or largest if consuming many strings such as concat).

For example, if a UDF accepts Utf8, LargeUtf8 and Utf8View types for input it should return data in the same type as the input (where feasible).

Currently I'm aware of the following UDF's that will consume Utf8View however do not return it:

- [ ] substrindex
- [ ] splitpart
- [X] translate
- [ ] replace
- [X] reverse
- [ ] rpad
- [ ] lpad (https://github.com/apache/datafusion/pull/20659)
- [x] repeat (https://github.com/apache/datafusion/pull/20645, https://github.com/apache/datafusion/pull/20612)
- [ ] overlay
- [X] lower
- [X] upper

It is possible there are others I'm unaware of

### Describe the solution you'd like

All udf's that return string data returns it in the same data type as the primary (or largest) argument type.

### Describe alternatives you've considered

_No response_

### Additional context

Searching for usages of the `utf8_to_str_type` function is a decent place to find udf's that have this issue. One should be careful with fixing that function itself (by adding a utf8view data type for example) as doing so will require all the usages of it to properly handle returning utf8View.

Contributor guide

Open the contributing guide

Research direction

Start by searching usages of the utf8_to_str_type function and reviewing the listed string UDFs: substrindex, splitpart, replace, rpad, overlay, and any others with the same issue. Check how each handles Utf8, LargeUtf8, and Utf8View inputs, and confirm that all applicable functions return the primary or largest input type without regressing existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
database
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.