apache / apache/datafusion

Add Backtraces to DataFusionError

Aperta
#5,283 2 commenti 1 reazione 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Rust
Stelle
9.3k
Fork
2.4k
Merge medio
3g 11h
PR unite (30g)
360

Descrizione

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

It is fairly common to get errors from DataFusion such as in https://github.com/apache/arrow-ballista/issues/665

```
2023-02-12T22:26:10.297254Z ERROR tokio-runtime-worker ThreadId(02) ballista_scheduler::scheduler_server::query_stage_scheduler: Error planning job t9P5iXR: DataFusionError(SchemaError(FieldNotFound { field: Column { relation: Some("part"), name: "p_brand" }, valid_fields: [Column { relation: None, name: "COUNT(DISTINCT partsupp.ps_suppkey)" }] }))
```

Which do not make it easy to understand the context in which the error was encountered. Typically the approach I follow is to use an interactive debugger, such as GDB, to catch the error. However, this requires having a local reproducer, which isn't always possible.

**Describe the solution you'd like**

Backtraces are a commonly used mechanism to automatically provide context on where an error was encountered. Whilst they have issues, especially in async contexts, they are better than the current situation where diagnosing such issues is non-trivial.

Adding a backtrace to the error variants that have corresponding `From` conversions, i.e. `ArrowError`, `ParquetError`, `AvroError`, `ObjectStore`, `IoError`, `SchemaError`, etc... would automatically capture the context in which this conversion is performed.

APIs can then use `DataFusionError::find_root` to get the leaf variant, and from that extract the backtrace if any.

This is predicated on the fact that `DataFusionError` is rarely used to implement control flow, and so the cost of obtaining a backtrace is only paid on exceptional paths.

**Describe alternatives you've considered**

**Additional context**
Add any other context or screenshots about the feature request here.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia leggendo DataFusionError::find_root e le conversioni From per ArrowError, ParquetError, AvroError, ObjectStore, IoError e SchemaError. Traccia come vengono rappresentate queste varianti e come i chiamanti possono accedere all'errore foglia. Il lavoro è completato quando le conversioni rilevanti catturano i backtrace e i chiamanti possono estrarli dalla variante foglia.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
rust
Ambito
backend
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
42/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.