Support accessing a map with non-literal, non-constant key
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
Accessing a map with a non-string key doesn't work (at least via DataFusion SQL) and is covered by https://github.com/apache/datafusion/issues/11785
Accessing a map with a literal string key works:
https://github.com/apache/datafusion/blob/5e1e693b6e30888d710aefc57be58ea25f62d34d/datafusion/sqllogictest/test_files/map.slt#L69-L72
Add support for accessing a map with non-literal key.
For example this should work (and also other cases where key cannot be constant-folded to a literal):
```
query ok
SELECT SUM(ints['by' || 'tes']) FROM data;
```
but it does not:
> `DataFusion error: Error during planning: Failed to coerce arguments to satisfy a call to 'array_element' function: coercion from [Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Int64, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false), Utf8] to the signature ArraySignature(ArrayAndIndexes(1)) failed No function matches the given name and argument types 'array_element(Map(Field { name: "entries", data_type: Struct([Field { name: "key", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "value", data_type: Int64, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, false), Utf8)'. You might need to add explicit type casts.
Candidate functions:
array_element(array, index`
Contributor guide
Research direction
The existing map case is in datafusion/sqllogictest/test_files/map.slt (L69-L72); start there and inspect the SQL planning path that produces the array_element coercion error. Add coverage for SELECT SUM(ints['by' || 'tes']) FROM data and confirm non-constant keys work without breaking literal-key access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- data-engineering, database
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100