apache / apache/datafusion-ballista

Stop showing very verbose streaming errors

Open
#224 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
2.1k
Forks
320
Avg merge
1d 22h
Merged PRs (30d)
66

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

This error is huge and not informative:

> ```2022-09-15T22:16:55.300475Z WARN tokio-runtime-worker ThreadId(31) ballista_executor::flight_service: Error streaming results: Status { code: Internal, message: "SendError(Ok(FlightData { flight_descriptor: None, data_header: [16, 0, 0, 0, 12, 0, 26, 0, 24, 0, 23, 0, 4, 0, 8, 0, 12, 0, 0, 0, 32, 0, 0, 0, 48, 23, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 4, 0, 10, 0, 20, 0, 12, 0, 8, 0, 4, 0, 10, 0, 0, 0, 116, 0, 0, 0, 12, 0, 0, 0, 81, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 81, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 12, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 16, 0, 0, 0, 0, 0, 0, 0, 68, 1, 0, 0, 0, 0, 0, 0, 88, 1, 0, 0, 0, 0, 0, 0, 11, 0, 0, 0, 0, 0, 0, 0, 104, 1, 0, 0 .....```

**Describe the solution you'd like**
Shorter error

**Describe alternatives you've considered**
None

**Additional context**
None

Contributor guide

Open the contributing guide

Research direction

Start at the ballista_executor::flight_service entry point and inspect where the "Error streaming results" warning formats its error. Trace the streaming error value to determine the intended concise message, then verify that the resulting warning no longer includes the large FlightData payload.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.