DataFusion should support casting strings such as "4e7" to decimal
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Is your feature request related to a problem or challenge?
DataFusion supports casting the string `4e7` to float but not to decimal. This is inconsistent with Postgres (and Apache Spark).
## Postgres
```
postgres=# select cast('4e7' as float);
float8
----------
40000000
(1 row)
postgres=# select cast('4e7' as decimal(10,2));
numeric
-------------
40000000.00
```
## Apache Spark
```
scala> spark.sql("select cast('4e7' as float)").show
+------------------+
|CAST(4e7 AS FLOAT)|
+------------------+
| 4.0E7|
+------------------+
scala> spark.sql("select cast('4e7' as decimal(10,2))").show
+--------------------------+
|CAST(4e7 AS DECIMAL(10,2))|
+--------------------------+
| 40000000.00|
+--------------------------+
```
## DataFusion
```
DataFusion CLI v37.0.0
❯ select cast('4e7' as float);
+-------------+
| Utf8("4e7") |
+-------------+
| 40000000.0 |
+-------------+
1 row in set. Query took 0.010 seconds.
❯ select cast('4e7' as decimal(10,2));
Arrow error: Cast error: Cannot cast string '4e7' to value of Decimal128(38, 10) type
```
### Describe the solution you'd like
_No response_
### Describe alternatives you've considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start by reproducing the `CAST('4e7' AS DECIMAL(10,2))` query in the DataFusion CLI and compare it with the existing float cast behavior. The work is done when scientific-notation strings such as `4e7` cast to decimal successfully and produce the expected value shown for Postgres and Apache Spark.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100