apache / apache/datafusion

More sensible handling of selecting camelcase columns

Open
#13,649 8 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

If I run the following query

```
SELECT userId FROM telemetry_user_events LIMIT 10;
```

against a table with the following schema
```

New Query
{
"fields": [
{
"name": "event",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
},
{
"name": "prompt",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
},
{
"name": "timestamp",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
},
{
"name": "timestamp_utc",
"data_type": {
"Timestamp": [
"Millisecond",
"UTC"
]
},
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
},
{
"name": "teamId",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
},
{
"name": "userId",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
},
{
"name": "query",
"data_type": "Utf8",
"nullable": true,
"dict_id": 0,
"dict_is_ordered": false,
"metadata": {}
}
],
"metadata": {}
}
```

I get the following error
```
Schema error: No field named userid. Valid fields are telemetry_user_events.event, telemetry_user_events.prompt, telemetry_user_events.timestamp, telemetry_user_events.timestamp_utc, telemetry_user_events.\"teamId\", telemetry_user_events.\"userId\", telemetry_user_events.query.\n 1: No field named userid. Valid fields are telemetry_user_events.event, telemetry_user_events.prompt, telemetry_user_events.timestamp, telemetry_user_events.timestamp_utc, telemetry_user_events.\"teamId\", telemetry_user_events.\"userId\", telemetry_user_events.query.
```

However if I wrap userId in quotes like so
```
SELECT "userId" FROM telemetry_user_events LIMIT 10;
```

there are no errors. but this seems unideal! all other sql query engines I know handle camelcase gracefully

### To Reproduce

Included in the above

### Expected behavior

no errors if i select a camelcase column without wrapping in quotes

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the query against the shown schema, comparing unquoted userId with quoted "userId". Trace the identifier-resolution path responsible for folding the unquoted name and add a regression test; done means selecting the camelcase column without quotes succeeds while quoted behavior remains valid.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.