apache / apache/datafusion

Nested struct field access on lambda parameters is unsupported (SQL and Substrait)

Open
#25,017 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

`e.field` doesn't resolve when `e` is a lambda parameter of struct type, in both the SQL frontend and the Substrait consumer path. Accessing the same nested field on an ordinary struct-typed column works fine in both -- this is specific to lambda-bound variables.

### To Reproduce

```
-- Works: whole lambda parameter, no dotted access. Real field names
-- (rule_number, rule_action) survive into the output type.
SELECT array_transform(
make_array(named_struct('rule_number', 1, 'rule_action', 'allow')),
e -> e
);

-- Fails: dotted access into the lambda parameter's own struct fields.
SELECT array_transform(
make_array(named_struct('rule_number', 1, 'rule_action', 'allow')),
e -> e.rule_action
);
```

The second query fails with `Schema error: No field named e.rule_action`. -- even purely in the SQL planner, no Substrait involved.

### Describe the solution you'd like

Support referencing a field in the struct type inside a lambda (in both paths)

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the two SQL examples and compare ordinary struct-field access with access through the lambda parameter. Trace field resolution in the SQL planner and the Substrait consumer path, both named in the issue. Done means dotted access to fields such as e.rule_action works in both paths while the existing whole-parameter query remains working.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.