apache / apache/datafusion

Improve consistency and documentation on error handling in in UDFs

Open
#11,618 12 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

When writing a new UDF, a developer needs to decide how to perform error management in functions that return `Result`, such as `return_type` and `invoke`. Looking at the existing codebase it is not obvious what are the error management best practices

# Errors in return type
- Regexp like uses an `plan_err` data types of args do not match the expected types https://github.com/apache/datafusion/blob/77311a5896272c7ed252d8cd53d48ec6ea7c0ccf/datafusion/functions/src/regex/regexplike.rs#L74 . Is this check redundant?

- Except expects two arguments but doesn't check the length, just the type https://github.com/apache/datafusion/blob/77311a5896272c7ed252d8cd53d48ec6ea7c0ccf/datafusion/functions-array/src/except.rs#L55.

# Errors in invoke

In the resize function, there is a check on argument lengths in invoke which is not present in the `return_type` function
https://github.com/apache/datafusion/blob/77311a5896272c7ed252d8cd53d48ec6ea7c0ccf/datafusion/functions-array/src/resize.rs#L27

The same function also returns `exec_err`, and `internal_datafusion_err`

### Describe the solution you'd like

As a developers of custom UDF I would like to know:
- what errors I need to check for and what are already checked by the planner (number of arguments?)
- what type of errors need to be raised in which conditions

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by comparing return_type and invoke in datafusion/functions/src/regex/regexplike.rs and datafusion/functions-array/src/except.rs and resize.rs, focusing on argument checks and the reported error variants. Review how planner validation relates to these functions; done means documenting the expected checks and error types, with any consistency changes covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.