apache / apache/datafusion-sqlparser-rs

Should the `generic` dialect support numeric literals with underscore separators?

Open
#2,432 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

Right now, the `generic` dialect (DataFusion's default) does not support numeric literals with separators:

```sql
DataFusion CLI v54.0.0
> select 1_000;
+------+
| _000 |
+------+
| 1 |
+------+
```

I'm wondering if the `generic` dialect should support this, just like Postgres/DuckDB/SQLite/Clickhouse, so DataFusion can use them directly without needing to switch to another dialect.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing `select 1_000;` in the DataFusion CLI and trace how the `generic` dialect tokenizes and parses numeric literals. Compare its behavior with the Postgres, DuckDB, SQLite, or ClickHouse dialects mentioned in the issue; done means the generic dialect accepts numeric separators without treating `_000` as a column name.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.