dolthub / dolthub/dolt

bad `double` conversion

Closed
#7,130 0 comments 0 reactions 1 assignee Claimed by @zachmu View on GitHub
analyzer bug correctness sql
Dominant language
Go
Stars
24.4k
Forks
873
Avg merge
1d 5h
Merged PRs (30d)
108

Description

We handle `doubles`/`float64s` just fine by themselves
```sql
tmp> select 1.7e+308;
+----------+
| 1.7e+308 |
+----------+
| 1.7e+308 |
+----------+
1 row in set (0.00 sec)
```

but struggle when performing arithmetic with other types
```sql
tmp> select 1.7e+308 + 0;
+--------------+
| 1.7e+308 + 0 |
+--------------+
| 0 |
+--------------+
1 row in set (0.00 sec)

tmp> select 1.7e+308 + 0.0;
+----------------+
| 1.7e+308 + 0.0 |
+----------------+
| 0.0 |
+----------------+
1 row in set (0.00 sec)

tmp> select 1.7e+308 + 1e10;
+-----------------+
| 1.7e+308 + 1e10 |
+-----------------+
| 10000000000 |
+-----------------+
1 row in set (0.00 sec)
```

MySQL returns `1.7e308` for all tests above

It's possible the issue stems from our `DecimalTypeMaxPrecision = 65`
```sql
tmp> select 1.7e64 + 123;
+-------------------------------------------------------------------+
| 1.7e64 + 123 |
+-------------------------------------------------------------------+
| 17000000000000000000000000000000000000000000000000000000000000123 |
+-------------------------------------------------------------------+
1 row in set (0.00 sec)

tmp> select 1.7e65 + 123;
+--------------+
| 1.7e65 + 123 |
+--------------+
| 123 |
+--------------+
1 row in set (0.00 sec)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.