apache / apache/datafusion-sqlparser-rs
support scientific notation when parsing numbers
- Dominant language
- Rust
- Stars
- 3.5k
- Forks
- 772
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 17
Description
`select 1e3` currently identifies 1 as a number and e3 as alias. Instead, it should return 1e3 as part of the string Number.
```sql
select 1e3; -- returns now e3 as identifier and 1 as number, but should return 1e3 as number
select 1e3a; -- should return 1e3 as number, and a as identifier
select 1e; -- should return 1e as the number (currently returns 1 as number, e as identifier)
select 1e.5; -- correctly returns syntax error
select .5e2; -- should return .5e2 as number, but currently returns .5 as number, e2 as identifier
```
Since the user of the library will get back a string representing the number anyway and must parse it, we could recognize the e notation and leave it as part of the string in Number.
For example, Rust can parse scientific notation to f64:
```rust
let v = ["1", "1e3", "1e", ".5e2", "1e-1"];
for s in v {
let _ = s
.parse::()
.map(|i| println!("parsed {} as i64: {}", s, i))
.or_else(|_| {
s.parse::()
.map(|f| println!("parsed {} as f64: {}", s, f))
})
.map_err(|_| println!("couldn't parse {}", s));
}
```
```
parsed 1 as i64: 1
parsed 1e3 as f64: 1000
couldn't parse 1e
parsed .5e2 as f64: 50
parsed 1e-1 as f64: 0.1
```
Only problem is that `1e` parses to `1` in Postgres, but it wouldn't in this case without special handling by the parsers. In any case, this is an argument in favor of returning scientific notation as part of the existing Number enum value.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue does not name a file or test. Start by locating the SQL lexer or numeric-token parsing entry point, then reproduce the listed inputs and add coverage for scientific notation, including trailing identifiers and exponent signs. Done means each example produces the requested Number or identifier tokens without changing the existing syntax error for `1e.5`.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- compilers, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100