apache / apache/arrow

[C++] ArrowInvalid: Failed to parse string: '2acv' as a scalar of type double

Open
#35,925 2 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Arrow raise `ArrowInvalid` exception when parsing non-numerical value to float failed even `allow_invalid_utf8` is true:
```python
str_arr = pa.array(["100", "200.22", "2acv"])
str_arr.cast(options=pc.CastOptions(target_type=pa.float64(), allow_invalid_utf8=True))
```

Exception:
```
---------------------------------------------------------------------------
ArrowInvalid Traceback (most recent call last)
Cell In[3], line 3
1 str_arr = pa.array(["100", "200.22", "2acv"])
2 print(pa.__version__)
----> 3 str_arr.cast(options=pc.CastOptions(target_type=pa.float64(), allow_invalid_utf8=True))

File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/array.pxi:935, in pyarrow.lib.Array.cast()

File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/compute.py:400, in cast(arr, target_type, safe, options, memory_pool)
398 else:
399 options = CastOptions.safe(target_type)
--> 400 return call_function("cast", [arr], options, memory_pool)

File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/_compute.pyx:572, in pyarrow._compute.call_function()

File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/_compute.pyx:367, in pyarrow._compute.Function.call()

File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/error.pxi:144, in pyarrow.lib.pyarrow_internal_check_status()

File ~/anaconda3/envs/py3.8/lib/python3.8/site-packages/pyarrow/error.pxi:100, in pyarrow.lib.check_status()

ArrowInvalid: Failed to parse string: '2acv' as a scalar of type double
```

### Component(s)

C++

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with the Python `Array.cast` and `pc.CastOptions` example, then trace the corresponding C++ cast implementation. Determine whether `allow_invalid_utf8` is intended to affect invalid numeric parsing, and verify the chosen behavior with a regression test for the `2acv` input.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.