apache / apache/datafusion

Better graphviz explains physical plans

Open
#17,598 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

@NGA-TRAN @robtandy and @gabotechs have a very neat execution plan visualization in the https://github.com/datafusion-contrib/datafusion-distributed

For example here is a taste from https://github.com/datafusion-contrib/datafusion-distributed/issues/117

Image

DataFusion includes graphviz output for now

```sql
> explain format graphviz select count(*) from 'benchmarks/data/hits.parquet' group by "URL";
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
| plan_type | plan |
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
| logical_plan | |
| | // Begin DataFusion GraphViz Plan, |
| | // display it online here: https://dreampuf.github.io/GraphvizOnline |
| | |
| | digraph { |
| | subgraph cluster_1 |
| | { |
| | graph[label="LogicalPlan"] |
| | 2[shape=box label="Projection: count(Int64(1)) AS count(*)"] |
| | 3[shape=box label="Aggregate: groupBy=[[benchmarks/data/hits.parquet.URL]], aggr=[[count(Int64(1))]]"] |
| | 2 -> 3 [arrowhead=none, arrowtail=normal, dir=back] |
| | 4[shape=box label="TableScan: benchmarks/data/hits.parquet projection=[URL]"] |
| | 3 -> 4 [arrowhead=none, arrowtail=normal, dir=back] |
| | } |
| | subgraph cluster_5 |
| | { |
| | graph[label="Detailed LogicalPlan"] |
| | 6[shape=box label="Projection: count(Int64(1)) AS count(*)\nSchema: [count(*):Int64]"] |
| | 7[shape=box label="Aggregate: groupBy=[[benchmarks/data/hits.parquet.URL]], aggr=[[count(Int64(1))]]\nSchema: [URL:Utf8View, count(Int64(1)):Int64]"] |
| | 6 -> 7 [arrowhead=none, arrowtail=normal, dir=back] |
| | 8[shape=box label="TableScan: benchmarks/data/hits.parquet projection=[URL]\nSchema: [URL:Utf8View]"] |
| | 7 -> 8 [arrowhead=none, arrowtail=normal, dir=back] |
| | } |
| | } |
| | // End DataFusion GraphViz Plan |
| | |
+--------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.001 seconds.
```

### Describe the solution you'd like

I would like a graphviz output for similar plans similar to the datafusion-distributed physical plan

Maybe we can port it directly 🤔

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with DataFusion's existing `EXPLAIN FORMAT GRAPHVIZ` output and compare it with the physical-plan visualization in datafusion-distributed issue 117. Trace how the current logical and detailed logical plans are rendered, then determine the corresponding physical-plan entry point; done means physical plans produce comparable Graphviz output.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.