apache / apache/datafusion

Parse real number literals as the Decimal type

Open
#12,817 4 comments 5 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

During the fix for [12655](https://github.com/apache/datafusion/issues/12655), I found that the root cause was that we parse real number literals as `f64`. When performing coercion between `f32` and `f64`, it can lead to significant precision problems.

As a solution, I proposed changing the default behavior to parse real number literals as Decimal. This not only helps avoid the precision issues between f32 and f64, but also aligns the default behavior with `Postgres` and `DuckDB`.

## Postgres
```
SELECT 1.3 as real , pg_typeof(1.3) as type;
```
| real | type |
|------|---------|
| 1.3 | numeric |

## DuckDB
```
SELECT 1.3 as real , typeof(1.3) as type;
```
| real | type |
|------|---------------|
| 1.3 | DECIMAL(2,1) |

### DataFusion
```
SELECT 1.3 as real , arrow_typeof(1.3) as type;
```
| real | type |
|------|---------|
| 1.3 | Float64 |

Fortunately, we have excellent support for Decimal, so the only change we need to make is setting the default value of `sql_parser.parse_float_as_decimal` to true. However, I believe this change could be breaking, as it alters the current behavior.

Therefore, it would be important to get broader community consensus before proceeding.

cc @alamb @jayzhan211 @jonahgao

### Describe the solution you'd like

change `sql_parser.parse_float_as_decimal` to `true`

### Describe alternatives you've considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating the sql_parser.parse_float_as_decimal setting and inspect how real-number literals are currently typed and tested. Compare the behavior with the PostgreSQL and DuckDB examples; done means the default produces Decimal values without regressing supported parsing or coercion behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.