apache / apache/arrow-rs

[arrow_json]`infer_json_schema()` infers `u64::MAX` as type `Int64`

Open Beginner friendly
#4,134 5 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

**Describe the bug**
[infer_json_schema()](https://docs.rs/arrow-json/latest/arrow_json/reader/fn.infer_json_schema.html) infers that `u64::MAX` is of type `Int64`, which is not correct as `i64` is not sufficient for `u64::MAX`.

**To Reproduce**
```toml
[package]
name = "rust"
version = "0.1.0"
edition = "2021"

[dependencies]
arrow = "37.0.0"
```
```rust
use arrow::json::reader::infer_json_schema;
use std::io::BufReader;

fn main() {
let json = r#"{"uint64_max": 18446744073709551615 }"#;
let mut buf_reader = BufReader::new(json.as_bytes());
let schema = infer_json_schema(&mut buf_reader, Some(1)).unwrap();

println!("{:?}", schema);
}
```
```shell
$ cargo r -q
Schema { fields: [Field { name: "uint64_max", data_type: Int64, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }], metadata: {} }
```

**Expected behavior**
`u64::MAX` should be inferred as type `UInt64`

**Additional context**
Related code snippet: https://github.com/apache/arrow-rs/blob/b9819219c8fb6c7e5c7336e3c2fcd86cb5befd98/arrow-json/src/reader/schema.rs#L444-L451

We can see that it defaults to `Int64` as long as it is not of type `f64`

Contributor guide

Open the contributing guide

Research direction

Start in arrow-json/src/reader/schema.rs around the referenced inference logic, then run the reproduction with u64::MAX to confirm the current schema. Trace how JSON numeric values are classified and add regression coverage for the boundary value. Done means infer_json_schema() reports UInt64 for u64::MAX without regressing other numeric types.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.