Remove `Wildcard` from `Expr`
- Lingua principale
- Rust
- Stelle
- 9.3k
- Fork
- 2.4k
- Merge medio
- 3g 11h
- PR unite (30g)
- 360
Descrizione
### Is your feature request related to a problem or challenge?
`Expr::Wildcard` and `Expr::QualifiedWildcard` are expressions that reference all columns. But it seems redundant that we don't need a special expr type to do that. This issue proposes to remove these two expr kinds.
### Describe the solution you'd like
Wildcard (`*`) can be expanded to concrete column lists when it appears. This manner seems viable in the three most common use cases I can come up with:
- (1) generate plan from SQL AST. Some scenarios already acted like this. E.g., `select * from table` will generate a projection with all fields, rather than a wildcard expr.
- (2) from `LogicalPlanBuilder` and (3) from `DataFrame`. In some aspects, these two entrances are the same. And both are strong typed (or strong schema-ed), which allows us to expand the wildcard immediately using the schema from current stage.
Besides this, `Expr::Wildcard` is not properly handled in the codebase, because it's not a "first class" expr. Take some functions as examples:
- `expr_to_columns`: from the correctness aspect, it should also count columns referenced by `Expr::Wildcard` and `Expr::QualifiedWildcard`.
- `create_physical_name`: this function requires all the wildcards to be expanded before calling it.
And I find another issue that related to `Wildcard`: https://github.com/apache/arrow-datafusion/issues/5473. The [solution](https://github.com/apache/arrow-datafusion/pull/5627) is to add an optimizer rule that expands all wildcards. This proposal is going to do something similar but in a more eager way.
### Describe alternatives you've considered
_No response_
### Additional context
Since `Expr` is widely used, we may need several versions to deprecate it (depending on the compatibility rule). We may ship this change step by step before fully removing these two variants
- change all the internal usage of wildcards (most of them are in aggr expr I suppose)
- change the public interface that may generate wildcard. Like https://github.com/apache/arrow-datafusion/issues/5473
- mark `Expr::Wildcard` and `Expr::QualifiedWildcard` as `#[deprecated]`
- fully remove them
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia tracciando la gestione dei wildcard attraverso il SQL AST, LogicalPlanBuilder e i punti di ingresso di DataFrame. Poi esamina expr_to_columns, create_physical_name e l’uso delle espressioni di aggregazione, insieme all’issue 5473 e alla PR 5627. Il lavoro è completato quando le espressioni wildcard vengono espanse nei punti di ingresso supportati e le due varianti di Expr sono deprecate o rimosse dopo l’aggiornamento degli utilizzi interni.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- rust, sql
- Ambito
- databases
- Tipo di issue
- Refactoring
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100