[Python] pyarrow large integer conversion
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
I have a json document that looks like this.
{"number": 123451234512345123451234512}
I then run the below code.
>>> from pyarrow.json import read_json
>>> pyarrow_table = read_json('pyarrow_test.json')
>>> pyarrow_table['number'][0].as_py().as_integer_ratio()
(123451234512345125900779520, 1)
notice the float that I get looks like it has been rounded or modified in some way.
Am I reading this file incorrectly or is there an issue with the conversion of this number?
**Reporter**: [Donald Freeman](https://issues.apache.org/jira/browse/ARROW-17393)
**Note**: *This issue was originally created as [ARROW-17393](https://issues.apache.org/jira/browse/ARROW-17393). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by reproducing the example with pyarrow.json.read_json and the supplied large-integer JSON value. Trace how the Python reader represents the number and determine whether the observed rounding is expected; done means the behavior is corrected or clearly covered by an agreed regression test or explanation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100