apache / apache/arrow

[C++] Allow converting strings to dates without using datetimes as an intermediate step

Open
#34,121 4 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

```
import pyarrow as pa, pyarrow.compute as C
x = pyarrow.array(['2008-01-01', '2008-01-02', '2008-01-03'])
```

This works fine:

```
>>> C.strptime(x, format = '%Y-%m-%d', unit = 's').cast('date32[day]')

[
2008-01-01,
2008-01-02,
2008-01-03
]
```

But this isn't allowed:

```
>>> C.strptime(x, format = '%Y-%m-%d', unit = 'day')
Traceback (most recent call last):
File "pyarrow/_compute.pyx", line 1515, in pyarrow._compute._StrptimeOptions._set_options
KeyError: 'day'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/home/arferk01/.local/lib/python3.10/site-packages/pyarrow/compute.py", line 251, in wrapper
options = _handle_options(func_name, options_class, options,
File "/home/arferk01/.local/lib/python3.10/site-packages/pyarrow/compute.py", line 214, in _handle_options
return options_class(*args, **kwargs)
File "pyarrow/_compute.pyx", line 1538, in pyarrow._compute.StrptimeOptions.__init__
File "pyarrow/_compute.pyx", line 1519, in pyarrow._compute._StrptimeOptions._set_options
File "pyarrow/_compute.pyx", line 619, in pyarrow._compute._raise_invalid_function_option
ValueError: "day" is not a valid time unit
```

It seems wasteful to have to compute a full seconds-level POSIX timestamp when the only thing you have and that you want is the date.

Related: #31254.

### Component(s)

Python

Contributor guide

Open the contributing guide

Research direction

Start with the Python option handling in pyarrow/_compute.pyx, especially StrptimeOptions and the C.strptime entry point shown in the report. Trace how time units are validated and locate the existing strptime tests. Done means date-valued parsing accepts the day unit directly and preserves the expected date results without requiring an intermediate datetime.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.