apache / apache/datafusion

Consider changing nullability of UDFs to depend on inputs by default

Open
#19,511 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

See: https://github.com/apache/datafusion/pull/19213#issuecomment-3635519486

Currently nullability is true by default for all UDFs; consider making it depend on input nullabilities (only nullable if inputs are nullable) since that looks to be the more common case. Exception functions would implement it back to true if this behaviour is not desirable.

This is likely to be a notable breaking change so discussion is needed first and the changes would need to be made with care.

Also related:

### Describe the solution you'd like

_No response_

### Describe alternatives you've considered

_No response_

### Additional context

Related:

- https://github.com/apache/datafusion/issues/18882

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked pull request comment and related issue 18882 to understand the proposed UDF nullability behavior and existing concerns. The work is not ready for implementation until the breaking-change discussion produces an agreed design; done would include that decision and a carefully scoped implementation plan.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.