apache / apache/datafusion

Query Planner Unable to Find Struct Fields with Capital Letters

Open
#16,648 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

If you create a table with a struct column where a field in the struct include a capital letter then the query planner will complain that it cannot find that field.

### To Reproduce

1. Create a JSON file named "test.json" with content:

{"thing": {"fooBar": "not going to find me"}}

2. Start the Datafusion cli
3. Execute "select thing.fooBar from 'test.json';"

Actual: The query will fail saying 'Error during planning: Field foobar not found in struct'.
Expected: The query executes returning 1 row with content "not going to find me"

Note: if you change the content of the file to:

{"thing": {"foobar": "not going to find me"}}

where the only change is "fooBar" to "foobar" then everything works.

### Expected behavior

Expected: The query executes returning 1 row with content "not going to find me"

Note: if you change the content of the file to:

{"thing": {"foobar": "not going to find me"}}

where the only change is "fooBar" to "foobar" then everything works.

### Additional context

Repros on 46.0.0 and 48.0.0

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue in the DataFusion CLI with the provided test.json and query. Trace the query-planner handling of struct field names to determine why fooBar becomes foobar. Done means the query returns the expected value for fooBar while the existing lowercase case continues to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.