Implement nested expression support in Substrait
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
_No response_
### Describe the solution you'd like
Here's a sample Substrait plan which uses a nested expression:
```
extension_uris {
extension_uri_anchor: 1
uri: "/unknown.yaml"
}
extension_uris {
extension_uri_anchor: 2
uri: "/functions_structs.yaml"
}
extensions {
extension_function {
extension_uri_reference: 1
function_anchor: 1
name: "min:i64"
}
}
extensions {
extension_function {
extension_uri_reference: 2
function_anchor: 2
name: "struct_extract:any_str"
}
}
relations {
root {
input {
project {
common {
emit {
output_mapping: 0
}
}
input {
aggregate {
common {
direct {
}
}
input {
project {
common {
emit {
output_mapping: 8
}
}
input {
read {
common {
direct {
}
}
base_schema {
names: "c_custkey"
names: "c_name"
names: "c_address"
names: "c_nationkey"
names: "c_phone"
names: "c_acctbal"
names: "c_mktsegment"
names: "c_comment"
struct {
types {
i64 {
nullability: NULLABILITY_NULLABLE
}
}
types {
string {
nullability: NULLABILITY_NULLABLE
}
}
types {
string {
nullability: NULLABILITY_NULLABLE
}
}
types {
i32 {
nullability: NULLABILITY_NULLABLE
}
}
types {
string {
nullability: NULLABILITY_NULLABLE
}
}
types {
decimal {
scale: 2
precision: 15
nullability: NULLABILITY_NULLABLE
}
}
types {
string {
nullability: NULLABILITY_NULLABLE
}
}
types {
string {
nullability: NULLABILITY_NULLABLE
}
}
nullability: NULLABILITY_REQUIRED
}
}
named_table {
names: "customer"
}
}
}
expressions {
nested {
struct {
fields {
selection {
direct_reference {
struct_field {
}
}
root_reference {
}
}
}
fields {
selection {
direct_reference {
struct_field {
field: 1
}
}
root_reference {
}
}
}
}
}
}
}
}
groupings {
}
measures {
measure {
function_reference: 1
phase: AGGREGATION_PHASE_INITIAL_TO_RESULT
output_type {
i64 {
nullability: NULLABILITY_REQUIRED
}
}
arguments {
value {
scalar_function {
function_reference: 2
arguments {
value {
selection {
direct_reference {
struct_field {
}
}
root_reference {
}
}
}
}
arguments {
value {
literal {
string: "c_custkey"
}
}
}
}
}
}
}
}
}
}
expressions {
selection {
direct_reference {
struct_field {
}
}
root_reference {
}
}
}
}
}
names: "result"
}
}
version {
minor_number: 52
producer: "my-producer"
}
```
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
No source file, test, or entry point is named. Start by locating DataFusion's Substrait plan import path and reproduce the provided nested-expression plan; done should mean this plan is accepted and nested expressions are handled without an import failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100