apache / apache/datafusion

Strengthen TypeSignature and Coercion rule.

Open
#10,507 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

Inspired from #10268. I have an idea to improve the current type signature and coercion design

## What is the current status

Given the function arguments, we check the arguments with the defined TypeSignature. `get_valid_types` is the function that calculates the possible valid types based on TypeSignature. After we get all the possible valid types, we find the **one** of the valid types among all the possible valid types. The core coercion rule is `coerced_from`. If every type in the valid types is coercible, it is the one we take.

## What is the issue of the current approach

**Given the signature is not well-supported**. We heavily rely on the coercion rule to get the expected types. We end up a complex coerce logic inside `coerced_from` function. It not only makes it hard to maintain (remove or change might cause the unknown issue to other functions), also contains duplicate (similar) logic to binary::coercion rule that is really confusing.
There are also cases that have coercion rule inside `return_type` of function which is not the expected place to fight with coercion.

## How to fix this

I think it is possible to improve the design of TypeSignature so that we can find the **one** possible valid types given the current types. The valid types we get are already coercible, so we don't need `coerced_from` function anymore!

After the change we can eliminate `coerced_from` function and only the binary::coercion rule is remain.

## Additional context
### Problematic examples
`array_concat` has signature variadic any, we have the coercion rule inside `return_type`.
`nullif` has coercion rule inside `return_type`

`coerced_from` has numeric coercion, list coercion, timestamp coercion, and even `comparison_binary_numeric_coercion` (which will be removed in #10268).

#10268 is the first step! 🚀

### Describe the solution you'd like

1. Support / Improve TypeSignature so we can get the only possible valid types given the arguments types we have.
2. Remove `coerced_from` function.

### Describe alternatives you've considered

I assume it is possible to find the only valid types given the argument types. If it is a false statement, we need to find another solution.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing TypeSignature, get_valid_types, and coerced_from, then compare their behavior with the binary::coercion rule. Review the array_concat and nullif return_type cases and the related #10268 work. Done means TypeSignature can select the single valid type for the argument types and coerced_from can be removed without retaining coercion logic in return_type.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.