apache / apache/datafusion

Epic: Simplify functions signature with LogicalType

Offen
#13,301 4 Kommentare 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen
enhancement EPIC
Vorherrschende Sprache
Rust
Sterne
9.3k
Forks
2.4k
Ø Merge
3 T. 11 Std.
Gemergte PRs (30 T.)
360

Beschreibung

### Is your feature request related to a problem or challenge?

Some functions signature is quite verbose, like

```rust
impl RegexpLikeFunc {
pub fn new() -> Self {
Self {
signature: Signature::one_of(
vec![
TypeSignature::Exact(vec![Utf8View, Utf8]),
TypeSignature::Exact(vec![Utf8View, Utf8View]),
TypeSignature::Exact(vec![Utf8View, LargeUtf8]),
TypeSignature::Exact(vec![Utf8, Utf8]),
TypeSignature::Exact(vec![Utf8, Utf8View]),
TypeSignature::Exact(vec![Utf8, LargeUtf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8View]),
TypeSignature::Exact(vec![LargeUtf8, LargeUtf8]),
TypeSignature::Exact(vec![Utf8View, Utf8, Utf8]),
TypeSignature::Exact(vec![Utf8View, Utf8View, Utf8]),
TypeSignature::Exact(vec![Utf8View, LargeUtf8, Utf8]),
TypeSignature::Exact(vec![Utf8, Utf8, Utf8]),
TypeSignature::Exact(vec![Utf8, Utf8View, Utf8]),
TypeSignature::Exact(vec![Utf8, LargeUtf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8View, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, LargeUtf8, Utf8]),
],
Volatility::Immutable,
),
}
}
}

Can replace it with `Signature::string(2, Volatility::Immutable)`
```

```rust
impl LPadFunc {
pub fn new() -> Self {
use DataType::*;
Self {
signature: Signature::one_of(
vec![
Exact(vec![Utf8View, Int64]),
Exact(vec![Utf8View, Int64, Utf8View]),
Exact(vec![Utf8View, Int64, Utf8]),
Exact(vec![Utf8View, Int64, LargeUtf8]),
Exact(vec![Utf8, Int64]),
Exact(vec![Utf8, Int64, Utf8View]),
Exact(vec![Utf8, Int64, Utf8]),
Exact(vec![Utf8, Int64, LargeUtf8]),
Exact(vec![LargeUtf8, Int64]),
Exact(vec![LargeUtf8, Int64, Utf8View]),
Exact(vec![LargeUtf8, Int64, Utf8]),
Exact(vec![LargeUtf8, Int64, LargeUtf8]),
],
Volatility::Immutable,
),
}
}
}

Can replace it with one of `Signature::coercible(string, int)` and `Signature::coercible(string, int, string)`
```

### Describe the solution you'd like

#13240 starts an attempt to bring logical type to function signature

There are more functions to be cleanup.
The example above can be replaced with TypeSiganture::String, TypeSiganture::Numeric, TypeSiganture::Coercible.

We might also need time related signature for time function.

Improve test coverage with these functions especially with different kinds of types would be great 👍

### The role of TypeSignature
TypeSignature used in function which is responsible for handling
1. length check
2. type checking and casting

**The functions behaviour follows Postgres, DuckDB or other well-designed database. Can check whether the result and coercion is consistent with them**.

If the result is consistent in both Postgres, DuckDB, we should follow them. Otherwise, we follow either of them.

For the casting rule, I think we can follow DuckDB's casting rule described [here](https://duckdb.org/docs/sql/data_types/typecasting.html)

TypeSignature should handle *implicit casting*

> [Implicit Casting](https://duckdb.org/docs/sql/data_types/typecasting.html#implicit-casting)
In many situations, the system will add casts by itself. This is called implicit casting. This happens for example when a function is called with an argument that does not match the type of the function, but can be casted to the desired type.

> Consider the function sin(DOUBLE). This function takes as input argument a column of type DOUBLE, however, it can be called with an integer as well: sin(1). The integer is converted into a double before being passed to the sin function.

> Implicit casts can only be added for a number of type combinations, and is generally only possible when the cast cannot fail. For example, an implicit cast can be added from INTEGER to DOUBLE – but not from DOUBLE to INTEGER.

Screenshot 2024-11-08 at 1 44 56 PM

# Tasks
- [ ] https://github.com/apache/datafusion/issues/19004
- [ ] https://github.com/apache/datafusion/issues/12725
- [ ] https://github.com/apache/datafusion/issues/18685
- [ ] https://github.com/apache/datafusion/issues/14761
- [x] https://github.com/apache/datafusion/issues/13552
- [ ] https://github.com/apache/datafusion/issues/13285
- [ ] https://github.com/apache/datafusion/issues/13371
- [ ] https://github.com/apache/datafusion/issues/10507
- [ ] https://github.com/apache/datafusion/issues/7580
- [x] https://github.com/apache/datafusion/issues/7225
- [ ] https://github.com/apache/datafusion/issues/6559
- [ ] https://github.com/apache/datafusion/issues/15817
- [x] https://github.com/apache/datafusion/issues/18092
- [ ] https://github.com/apache/datafusion/issues/14760
- [ ] https://github.com/apache/datafusion/issues/14763
- [ ] https://github.com/apache/datafusion/issues/12698
- [x] https://github.com/apache/datafusion/issues/19438
- [ ] https://github.com/apache/datafusion/issues/19458
- [ ] https://github.com/apache/datafusion/issues/24133
- [x] https://github.com/apache/datafusion/issues/24135
- [ ] https://github.com/apache/datafusion/issues/24136
- [ ] https://github.com/apache/datafusion/issues/24141

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Lies zuerst #13240 und die Definitionen von Signature/TypeSignature; wähle anhand der Links zu den nicht abgehakten Aufgaben einen einzelnen Funktionsbereich aus und vergleiche sein Verhalten mit Postgres oder DuckDB. Erledigt bedeutet, dass die ausgewählten Signaturen logische Typen verwenden, das Verhalten impliziter Casts durch Tests abgedeckt ist und das Ergebnis dem Verhalten der ausgewählten Datenbank entspricht.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
rust
Bereich
databases
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.