[C++] strptime rolls-over dates not in range for current month
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
I noticed some potentially unexpected behaviour when converting from string to date. Days that are out of bounds for the given month are rolled over into the following month.
I think the expected behaviour would be to either error (Python) or return NULL/NA (R), but not to roll over dates in the following month.
```r
library(arrow, warn.conflicts = FALSE)
library(lubridate, warn.conflicts = FALSE)
library(dplyr, warn.conflicts = FALSE)
df <- tibble::tibble(string_date = "1999-02-30")
# base R returns NA
df %>%
mutate(date = strptime(string_date, format = "%Y-%m-%d"))
#> # A tibble: 1 × 2
#> string_date date
#>
#> 1 1999-02-30 NA
# arrow rolls over the 30th of February into the 2nd of March
df %>%
arrow_table() %>%
mutate(date = strptime(string_date, format = "%Y-%m-%d")) %>%
collect()
#> # A tibble: 1 × 2
#> string_date date
#>
#> 1 1999-02-30 1999-03-02 00:00:00
```
Thanks Alenka, Joris and Rok for helping me with the Python examples:
pandas:
```python
>>> import pandas as pd
>>> pd.to_datetime("1999-02-30", format="%Y-%m-%d")
...
ValueError: time data 1999-02-30 doesn't match format specified
```
datetime:
```python
>>> import datetime
>>> from datetime import datetime
>>> datetime.strptime("1999-02-30", "%Y-%m-%d")
...
ValueError: day is out of range for month
```
arrow:
```python
>>> import pyarrow.compute as pc
>>> print(pc.strptime("1999-02-30", format="%Y-%m-%d", unit="s"))
1999-03-02 00:00:00
```
**Reporter**: [Dragoș Moldovan-Grünfeld](https://issues.apache.org/jira/browse/ARROW-15948) / @dragosmg
**Watchers**: [Rok Mihevc](https://issues.apache.org/jira/browse/ARROW-15948) / @rok
#### Related issues:
- [[C++] Add error handling option to StrptimeOptions](https://github.com/apache/arrow/issues/20115) (is related to)
**Note**: *This issue was originally created as [ARROW-15948](https://issues.apache.org/jira/browse/ARROW-15948). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Reproduce the invalid-date case through pyarrow.compute.strptime using "1999-02-30" and format "%Y-%m-%d". Then read the C++ strptime implementation and related issue #20115 about StrptimeOptions, establish the intended invalid-date behavior, and add regression coverage showing that out-of-range days no longer silently roll into the next month.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100