apache / apache/datafusion

Epic: Simplify functions signature with LogicalType

Aperta
#13,301 4 commenti 1 reazione 0 assegnatari Vedi su GitHub
enhancement EPIC
Lingua principale
Rust
Stelle
9.3k
Fork
2.4k
Merge medio
3g 11h
PR unite (30g)
360

Descrizione

### Is your feature request related to a problem or challenge?

Some functions signature is quite verbose, like

```rust
impl RegexpLikeFunc {
pub fn new() -> Self {
Self {
signature: Signature::one_of(
vec![
TypeSignature::Exact(vec![Utf8View, Utf8]),
TypeSignature::Exact(vec![Utf8View, Utf8View]),
TypeSignature::Exact(vec![Utf8View, LargeUtf8]),
TypeSignature::Exact(vec![Utf8, Utf8]),
TypeSignature::Exact(vec![Utf8, Utf8View]),
TypeSignature::Exact(vec![Utf8, LargeUtf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8View]),
TypeSignature::Exact(vec![LargeUtf8, LargeUtf8]),
TypeSignature::Exact(vec![Utf8View, Utf8, Utf8]),
TypeSignature::Exact(vec![Utf8View, Utf8View, Utf8]),
TypeSignature::Exact(vec![Utf8View, LargeUtf8, Utf8]),
TypeSignature::Exact(vec![Utf8, Utf8, Utf8]),
TypeSignature::Exact(vec![Utf8, Utf8View, Utf8]),
TypeSignature::Exact(vec![Utf8, LargeUtf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, Utf8View, Utf8]),
TypeSignature::Exact(vec![LargeUtf8, LargeUtf8, Utf8]),
],
Volatility::Immutable,
),
}
}
}

Can replace it with `Signature::string(2, Volatility::Immutable)`
```

```rust
impl LPadFunc {
pub fn new() -> Self {
use DataType::*;
Self {
signature: Signature::one_of(
vec![
Exact(vec![Utf8View, Int64]),
Exact(vec![Utf8View, Int64, Utf8View]),
Exact(vec![Utf8View, Int64, Utf8]),
Exact(vec![Utf8View, Int64, LargeUtf8]),
Exact(vec![Utf8, Int64]),
Exact(vec![Utf8, Int64, Utf8View]),
Exact(vec![Utf8, Int64, Utf8]),
Exact(vec![Utf8, Int64, LargeUtf8]),
Exact(vec![LargeUtf8, Int64]),
Exact(vec![LargeUtf8, Int64, Utf8View]),
Exact(vec![LargeUtf8, Int64, Utf8]),
Exact(vec![LargeUtf8, Int64, LargeUtf8]),
],
Volatility::Immutable,
),
}
}
}

Can replace it with one of `Signature::coercible(string, int)` and `Signature::coercible(string, int, string)`
```

### Describe the solution you'd like

#13240 starts an attempt to bring logical type to function signature

There are more functions to be cleanup.
The example above can be replaced with TypeSiganture::String, TypeSiganture::Numeric, TypeSiganture::Coercible.

We might also need time related signature for time function.

Improve test coverage with these functions especially with different kinds of types would be great 👍

### The role of TypeSignature
TypeSignature used in function which is responsible for handling
1. length check
2. type checking and casting

**The functions behaviour follows Postgres, DuckDB or other well-designed database. Can check whether the result and coercion is consistent with them**.

If the result is consistent in both Postgres, DuckDB, we should follow them. Otherwise, we follow either of them.

For the casting rule, I think we can follow DuckDB's casting rule described [here](https://duckdb.org/docs/sql/data_types/typecasting.html)

TypeSignature should handle *implicit casting*

> [Implicit Casting](https://duckdb.org/docs/sql/data_types/typecasting.html#implicit-casting)
In many situations, the system will add casts by itself. This is called implicit casting. This happens for example when a function is called with an argument that does not match the type of the function, but can be casted to the desired type.

> Consider the function sin(DOUBLE). This function takes as input argument a column of type DOUBLE, however, it can be called with an integer as well: sin(1). The integer is converted into a double before being passed to the sin function.

> Implicit casts can only be added for a number of type combinations, and is generally only possible when the cast cannot fail. For example, an implicit cast can be added from INTEGER to DOUBLE – but not from DOUBLE to INTEGER.

Screenshot 2024-11-08 at 1 44 56 PM

# Tasks
- [ ] https://github.com/apache/datafusion/issues/19004
- [ ] https://github.com/apache/datafusion/issues/12725
- [ ] https://github.com/apache/datafusion/issues/18685
- [ ] https://github.com/apache/datafusion/issues/14761
- [x] https://github.com/apache/datafusion/issues/13552
- [ ] https://github.com/apache/datafusion/issues/13285
- [ ] https://github.com/apache/datafusion/issues/13371
- [ ] https://github.com/apache/datafusion/issues/10507
- [ ] https://github.com/apache/datafusion/issues/7580
- [x] https://github.com/apache/datafusion/issues/7225
- [ ] https://github.com/apache/datafusion/issues/6559
- [ ] https://github.com/apache/datafusion/issues/15817
- [x] https://github.com/apache/datafusion/issues/18092
- [ ] https://github.com/apache/datafusion/issues/14760
- [ ] https://github.com/apache/datafusion/issues/14763
- [ ] https://github.com/apache/datafusion/issues/12698
- [x] https://github.com/apache/datafusion/issues/19438
- [ ] https://github.com/apache/datafusion/issues/19458
- [ ] https://github.com/apache/datafusion/issues/24133
- [x] https://github.com/apache/datafusion/issues/24135
- [ ] https://github.com/apache/datafusion/issues/24136
- [ ] https://github.com/apache/datafusion/issues/24141

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Leggi prima #13240 e le definizioni di Signature/TypeSignature; usa i link alle attività non selezionate per scegliere una singola area funzionale e confrontane il comportamento con Postgres o DuckDB. Il lavoro è completato quando le signature selezionate usano tipi logici, il comportamento dell’implicit casting è coperto dai test e il risultato corrisponde al comportamento del database scelto.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
rust
Ambito
databases
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.