apache / apache/datafusion

Implement protobuf serialization for Expr::ScalarSubquery(_) | Expr::InSubquery { .. } | Expr::Exists { .. }

Offen
#4,337 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Rust
Sterne
9.3k
Forks
2.4k
Ø Merge
3 T. 11 Std.
Gemergte PRs (30 T.)
360

Beschreibung

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
Note depends on:
- [x] https://github.com/apache/arrow-datafusion/issues/4338

In IOx, when a user issued an unsupported subquery, they got a very confusing message about serialization

```
❯ select count(*) from foo where exists (select column1 from foo);
Error during planning: Error encoding expr as protobuf: General error: Proto serialization error: Expr::ScalarSubquery(_) | Expr::InSubquery { .. } | Expr::Exists { .. }#6195
```

This is because the EXISTS predicate could not be sent / serialized. If it could have been, the code in the physical plan would have said "does not support logical exists (SUBQUERY)"

**Describe the solution you'd like**
I would like to implement the protobuf serialization for such subqueries. This would allow the user to see the real error

```
❯ select count(*) from foo where exists (select column1 from foo);
NotImplemented("Physical plan does not support logical expression EXISTS ()")
```

https://github.com/apache/arrow-datafusion/blob/d355f69aae2cc951cfd021e5c0b690861ba0c4ac/datafusion/proto/src/to_proto.rs#L850-L853

However in order to do so we would likely have to serialize entire LogicalPlans -- https://github.com/apache/arrow-datafusion/issues/4338

**Describe alternatives you've considered**
We could put a specific error in IOx for this error but I would prefer to implement the protobuf serialization because it will stay in sync with DataFusions features (e.g. when datafusion does support this kind of query, we won't have to remember to remove the IOx specific check)

**Additional context**
Add any other context or screenshots about the feature request here.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne bei datafusion/proto/src/to_proto.rs#L850-L853 und lies den Dependency-Issue #4338 über das Serialisieren vollständiger LogicalPlans. Reproduziere die EXISTS-Abfrage aus dem Issue, um den aktuellen Serialisierungsfehler zu beobachten. Das Ziel ist erreicht, wenn Subqueries protobuf-serialisiert werden können und die Abfrage stattdessen den Fehler physical-plan unsupported-expression error meldet.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
rust
Bereich
backend
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.