apache / apache/datafusion-sqlparser-rs

support scientific notation when parsing numbers

Open
#610 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
3.5k
Forks
772
Avg merge
4d 9h
Merged PRs (30d)
17

Description

`select 1e3` currently identifies 1 as a number and e3 as alias. Instead, it should return 1e3 as part of the string Number.

```sql
select 1e3; -- returns now e3 as identifier and 1 as number, but should return 1e3 as number
select 1e3a; -- should return 1e3 as number, and a as identifier
select 1e; -- should return 1e as the number (currently returns 1 as number, e as identifier)
select 1e.5; -- correctly returns syntax error
select .5e2; -- should return .5e2 as number, but currently returns .5 as number, e2 as identifier
```

Since the user of the library will get back a string representing the number anyway and must parse it, we could recognize the e notation and leave it as part of the string in Number.

For example, Rust can parse scientific notation to f64:
```rust
let v = ["1", "1e3", "1e", ".5e2", "1e-1"];
for s in v {
let _ = s
.parse::()
.map(|i| println!("parsed {} as i64: {}", s, i))
.or_else(|_| {
s.parse::()
.map(|f| println!("parsed {} as f64: {}", s, f))
})
.map_err(|_| println!("couldn't parse {}", s));
}
```

```
parsed 1 as i64: 1
parsed 1e3 as f64: 1000
couldn't parse 1e
parsed .5e2 as f64: 50
parsed 1e-1 as f64: 0.1
```
Only problem is that `1e` parses to `1` in Postgres, but it wouldn't in this case without special handling by the parsers. In any case, this is an argument in favor of returning scientific notation as part of the existing Number enum value.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue does not name a file or test. Start by locating the SQL lexer or numeric-token parsing entry point, then reproduce the listed inputs and add coverage for scientific notation, including trailing identifiers and exponent signs. Done means each example produces the requested Number or identifier tokens without changing the existing syntax error for `1e.5`.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
compilers, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.