[C++] ArrowInvalid: Failed to parse string: '2acv' as a scalar of type double
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
### Describe the bug, including details regarding any error messages, version, and platform.
Arrow raise `ArrowInvalid` exception when parsing non-numerical value to float failed even `allow_invalid_utf8` is true:
```python
str_arr = pa.array(["100", "200.22", "2acv"])
str_arr.cast(options=pc.CastOptions(target_type=pa.float64(), allow_invalid_utf8=True))
```
Exception:
```
---------------------------------------------------------------------------
ArrowInvalid Traceback (most recent call last)
Cell In[3], line 3
1 str_arr = pa.array(["100", "200.22", "2acv"])
2 print(pa.__version__)
----> 3 str_arr.cast(options=pc.CastOptions(target_type=pa.float64(), allow_invalid_utf8=True))
File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/array.pxi:935, in pyarrow.lib.Array.cast()
File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/compute.py:400, in cast(arr, target_type, safe, options, memory_pool)
398 else:
399 options = CastOptions.safe(target_type)
--> 400 return call_function("cast", [arr], options, memory_pool)
File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/_compute.pyx:572, in pyarrow._compute.call_function()
File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/_compute.pyx:367, in pyarrow._compute.Function.call()
File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/error.pxi:144, in pyarrow.lib.pyarrow_internal_check_status()
File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/error.pxi:100, in pyarrow.lib.check_status()
ArrowInvalid: Failed to parse string: '2acv' as a scalar of type double
```
### Component(s)
C++
Contributor guide
Research direction
Reproduce the issue with the Python `Array.cast` and `pc.CastOptions` example, then trace the corresponding C++ cast implementation. Determine whether `allow_invalid_utf8` is intended to affect invalid numeric parsing, and verify the chosen behavior with a regression test for the `2acv` input.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100