crate / crate/crate-python

NUMERIC reads: full digits arrive, the decode drops them

Open
#826 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
85
Forks
34
Avg merge
3d 1h
Merged PRs (30d)
4

Description

The read-side follow-up promised in crate/sqlalchemy-cratedb#300, with the measurement #652 was missing. As @matriv said there, the full number reaches the client; the digits are dropped by this driver's decode.

On CrateDB 6.4.2, with `1.234567890123456789012345` stored in a `NUMERIC(38, 24)` column, the raw `/_sql` response carries every digit:

```json
{"cols":["n"],"rows":[[1.234567890123456789012345]],"rowcount":1,"duration":105.078}
```

`cursor.fetchone()[0]` on the same query returns `1.2345678901234567`, a float. The response is decoded with `orjson.loads` ([http.py line 244](https://github.com/crate/crate-python/blob/a61a3e5f30fafce3956b800a3d6f9a256aa0fac1/src/crate/client/http.py#L244)), which parses every JSON number to a float64 and has no `parse_float` hook; stdlib `json.loads(raw, parse_float=Decimal)` on those bytes returns `Decimal('1.234567890123456789012345')`.

Writes are already exact, since `Decimal` goes out as a string (#751). A stdlib decode with `parse_float=Decimal` would make reads match, but it changes the returned type for every float column and gives up orjson's speed, so it likely wants to be a connection option rather than the default.

Contributor guide

Open the contributing guide

Research direction

Start in src/crate/client/http.py at line 244 and reproduce the NUMERIC example against CrateDB 6.4.2. Compare the current orjson decode with stdlib json.loads using parse_float=Decimal, then determine how a connection option should expose exact numeric reads while preserving existing float behavior; done means the reported digits are retained and the option is covered by verification.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.