Preserve float-ness when encoding floats
- Dominant language
- Gleam
- Stars
- 148
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
In one of my projects I have a browser and a server where I generate JSON payloads with `gleam/json` before sending them to the server, which will reconstruct the Gleam type.
Everything was fine until I discovered that one of my JSON requests was failing to be decoded on my server.
What happened was that in one of my requests, I encoded in JavaScript with `json.float` a float that was representable as an integer, and `json.to_string` produced a JSON payload with a number without any `.0` suffix.
So when the JSON arrived on the Erlang server, `json.parse` saw a number like `8` in the payload and generated a `Dynamic` holding an `Int`. Yet in the JSON spec it is totally valid for a float to be represented without the `.0`, and since the `Dynamic` here is not a `Float`, my decoder fails.
Here is a simple Erlang/Gleam code snippet that reproduces what happens:
```gleam
import gleam/dynamic/decode
import gleam/json
pub type Hoi {
Hoi(number: Float)
}
// Generated by the gleam LSP
pub fn hoi_to_json(hoi: Hoi) -> json.Json {
let Hoi(number:) = hoi
json.object([
#("number", json.float(number)),
])
}
// Generated by the gleam LSP
pub fn hoi_decoder() -> decode.Decoder(Hoi) {
use number <- decode.field("number", decode.float)
decode.success(Hoi(number:))
}
pub fn main() -> Nil {
//! Imagine we are on the JS side
let hoi = Hoi(number: 8.0)
// On the Erlang side `json.float` appends `.0` but JavaScript does not
let payload = json.to_string(hoi_to_json(hoi))
// This line reproduces this behavior
let payload = "{\"number\":8}"
//! Imagine we send this over HTTP
echo payload
//! Now we are on the Erlang side with the received payload
let hoi2 = json.parse(payload, hoi_decoder())
case hoi2 {
Ok(hoi2) -> {
assert hoi == hoi2
}
Error(err) -> {
// UnableToDecode([DecodeError("Float", "Int", ["number"])])
echo err
panic as "Failed to decode json"
}
}
echo "OK"
Nil
}
```
For now, my workaround is to manually edit the decoder generated by the LSP to use `decode.one_of`:
```gleam
import gleam/dynamic/decode
import gleam/int
pub fn hoi_decoder() -> decode.Decoder(Hoi) {
use number <- decode.field(
"number",
decode.one_of(decode.float, [decode.map(decode.int, int.to_float)]),
)
decode.success(Hoi(number:))
}
```
Sadly, this means it is not the default behavior, and people who create decoders for JSON and want to work with `Float` need to be careful not to forget (or need to already know) that they have to do this.
That said, changing the default behavior may not be what people want in other use cases, since sometimes you want to ensure the data has an exact type for memory reasons.
One might also think this issue could be solved by making `json.float` on the JS side always emit a `.0` suffix, but this would differ from JavaScript's native `JSON.stringify` behavior:
```js
JSON.stringify(8.0)
"8"
```
It also would not solve the issue if someone builds a public API where users send raw JSON directly.
Finally, it is hard to think of a solution at the `json.parse` level, since with the stdlib `dynamic` library we cannot express something that is either an `Int` or could be coerced to a `Float`.
The best thing that could be done is adding a helper in `gleam/dynamic/decode` that calls `decode.one_of`, but we would still be left with LSP generated decoders that are not JSON compatible by default and I am not sure whether generating them with a coercion mechanism would be the behavior people expect by default.
Anyway, it is a tricky issue, and @lpil asked me to open an issue on the `gleam/json` package so here I am.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the behavior of `json.float`, `json.to_string`, and `json.parse`, then compare it with `decode.float` and the LSP-generated decoder shown in the issue. No files or tests are named. Done requires an agreed behavior for integer-form JSON numbers decoded as floats without breaking cases that require exact numeric types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- erlang, javascript
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100