Sort key over get_field(named_struct(...), f) is evaluated against the materialized struct instead of the base column
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
Follow-up from #22239. For a query that sorts by a struct field, where the table is `WITH ORDER (a + b)` so a `SortExec` is genuinely required:
```sql
EXPLAIN SELECT named_struct('a', a, 'b', b) AS s FROM ordered ORDER BY s['a'];
```
**Plan before #22239** — `get_field` sort key extracted into a flat scan column, recovery projection hides it:
```
01)ProjectionExec: expr=[s@0 as s]
02)--SortExec: expr=[__datafusion_extracted_1@1 ASC NULLS LAST], preserve_partitioning=[false]
03)----DataSourceExec: projection=[named_struct(a, a@0, b, b@1) as s, get_field(named_struct(a, a@0, b, b@1), a) as __datafusion_extracted_1], ...
```
**Plan after #22239** — simpler, but the sort key is a struct-field extraction over the materialized struct:
```
01)SortExec: expr=[get_field(s@0, a) ASC NULLS LAST], preserve_partitioning=[false]
02)--DataSourceExec: projection=[named_struct(a, a@0, b, b@1) as s], ...
```
**Ideal plan** — sort on the base column, build the struct afterward:
```
01)ProjectionExec: expr=[named_struct(a, a@0, b, b@1) as s]
02)--SortExec: expr=[a@0 ASC NULLS LAST]
03)----DataSourceExec: projection=[a, b]
```
Contributor guide
Research direction
Start by reproducing the provided EXPLAIN query against the ordered table and compare the current plan with the ideal plan. Trace how the sort key for get_field(named_struct(...), 'a') is extracted and when the struct is materialized; done means SortExec orders by the base column and the projection builds the struct afterward.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100