grafana / grafana/google-bigquery-datasource

NUMERIC / BIGNUMERIC values lose precision (converted via float64)

Open
#502 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
40
Forks
24
Avg merge
3d 21h
Merged PRs (30d)
8

Description

### Summary

`NUMERIC` and `BIGNUMERIC` columns are returned through a `float64` conversion, so values with more than ~15–16 significant digits are silently rounded/corrupted. `NUMERIC` is an exact decimal (38 digits, scale 9) and `BIGNUMERIC` is 76 digits, but `float64` has only ~15–16 significant decimal digits.

### Location

`pkg/bigquery/driver/utils.go`:

```go
case "NUMERIC", "BIGNUMERIC":
conv, _ := v.(*big.Rat).Float64()
return conv, nil
```

The exact `*big.Rat` is downcast to `float64` (and the `exact` bool returned by `Float64()` is discarded).

### Evidence

Querying these through the plugin and comparing the returned value to the exact input:

| query | exact value | returned by plugin |
|---|---|---|
| `CAST('12345678901234567.123456789' AS NUMERIC)` | 12345678901234567.123456789 | `12345678901234568` |
| `CAST('9999999999999999' AS NUMERIC)` | 9999999999999999 | `10000000000000000` |
| `CAST('1234567890123456789012345.123' AS BIGNUMERIC)` | 1234567890123456789012345.123 | `1.2345678901234568e+24` |

**Control:** casting the same NUMERIC to `STRING` returns it exactly (`12345678901234567.123456789`), confirming BigQuery holds the exact value and the loss is introduced by the plugin's `float64` conversion.

Note the second row: an exact 16-digit integer (`9999999999999999`) comes back as `10000000000000000` — off by one, not merely a fractional rounding.

### Impact

Financial amounts, large IDs, and high-precision measurements stored as `NUMERIC`/`BIGNUMERIC` are silently corrupted, with no error. The loss is unrecoverable downstream.

### Suggested fix

Return the exact value losslessly — e.g. as a string via `(*big.Rat).FloatString(scale)` using the column's scale — instead of `float64`.

This is a behavior change (the field would surface as string rather than number), so it may warrant gating or applying it only when the value can't be represented exactly. Happy to discuss the approach and open a PR.

Contributor guide

Open the contributing guide

Research direction

Start in pkg/bigquery/driver/utils.go at the NUMERIC/BIGNUMERIC conversion case, then inspect how its returned value is exposed by the plugin. Reproduce the listed precision-loss examples and determine the agreed lossless representation, including the behavior change from numeric to string. Done means exact NUMERIC and BIGNUMERIC values no longer undergo silent float64 corruption.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.