Scalars are too verbose in column name output
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 362
Description
### Is your feature request related to a problem or challenge?
When parsing scalar expressions, DataFusion makes queries quite complicated:
```sql
> select 3, array_length([1, 2, 4, 5, 10000, 2.4]);;
+----------+-----------------------------------------------------------------------------------------+
| Int64(3) | array_length(make_array(Int64(1),Int64(2),Int64(4),Int64(5),Int64(10000),Float64(2.4))) |
+----------+-----------------------------------------------------------------------------------------+
| 3 | 6 |
+----------+-----------------------------------------------------------------------------------------+
```
Such detailed comparison isn't needed in most cases and adds unnecessary cognitive complexity for developers. Moreover, it also breaks the original query - if you put `select Int64(3)`, you'll get `Error during planning: Invalid function 'int64'`
### Describe the solution you'd like
In most cases, the simplest version will be enough. On the example above it is:
```sql
> select 3, array_length([1, 2, 4, 5, 10000, 2.4]);
+---+---------------------------------------------+
| 3 | array_length(make_array(1,2,4,5,10000,2.4)) |
+---+---------------------------------------------+
| 3 | 6 |
+---+---------------------------------------------+
```
IMO there are three ways how to fix this problem:
1. Just remove `:?` here for all cases
https://github.com/apache/datafusion/blob/0ff89844a2f2c7a3bbcaff995db7e464deaeb997/datafusion/expr/src/expr.rs#L2951
This will make queries simpler everywhere. The downside is that we lose some information, e.g., type info in the shell (which one can argue isn't needed and can be checked separately)
2. Disable verbose as a `ConfigOptions` param and make it changeable (via datafusion-cli or sdk).
3. Use short names only if parsing back preserves the correct type. For example, when formatting `Int64(0)`, output `0` (since parsing `0` results in `Int64(0)`). When formatting `Int32(0)`, keep `Int32(0)`.
### Additional context
May be quite easy to update after we finish https://github.com/apache/datafusion/issues/15178
Contributor guide
Research direction
Start with the scalar expression formatting code at datafusion/expr/src/expr.rs around line 2951, then read issue #15178 for the stated dependency. Decide which of the three formatting approaches preserves the needed type information, update the column-name output accordingly, and verify that the displayed SQL no longer uses unnecessarily verbose scalar names.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100