deprecating `return_type` in favor of `return_fields_from_args`
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
Having `return_type` does not really reflect the UDF actual nullability.
the following problems exists when UDF wrongly report their nullability:
## Disallow optimization based on the expressions nullability.
For example lets assume I have 2 columns:
1. `col1` which is i64 without nulls
2. `col2` which is i64 with nulls
and the expression `ceil` does not does not implement `return_fields_from_args`, so by default the `nullability` is `true`
can't optimize the following query and
this means that I can can't optimize this query
```
select coalesce(ceil('col1'), 'col2') from tbl;
```
to be this:
```
select ceil('col1') from tbl;
```
because `ceil` says that the nullability is true even though the input expression is not nullable
the same can be for not able to remove the expression `array_remove_all(array, null)` if the array expression report that it is nullable while it is not
## Wrong types
creating the wrong type:
```
> select arrow_typeof(make_array(1));
+------------------------------------------------------------------------------------------------------------------+
| arrow_typeof(make_array(Int64(1))) |
+------------------------------------------------------------------------------------------------------------------+
| List(Field { name: "item", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }) |
+------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.002 seconds.
```
should be nullable `false` but because `make_array` implement `return_type` it does not get that information
Contributor guide
Research direction
Start by locating the UDF implementations that use `return_type`, especially `ceil` and `make_array`, and compare them with `return_fields_from_args`. Trace the nullability and type inference tests, then confirm that affected UDFs derive those fields from their arguments and that the existing SQL examples produce the expected results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- backend-api-design, databases, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100