apache / apache/arrow

[C++][Python] Wrong result when converting time zones after 2038

Open
#36,110 8 comments 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the bug, including details regarding any error messages, version, and platform.

```python
import pyarrow as pa
import pyarrow.compute as pc

from datetime import datetime, timezone
from zoneinfo import ZoneInfo

string = '2038-04-01 03:00:00.000000'

dt = datetime.fromisoformat(string)
dt = dt.replace(tzinfo=ZoneInfo('America/Boise'))
tz = ZoneInfo('UTC')
converted_dt = dt.astimezone(tz)
print(converted_dt)

ts = pc.assume_timezone(pa.array([datetime(2038, 4, 1, 3)]), timezone='America/Boise')
print(ts)
```

The above prints
```
2038-04-01 09:00:00+00:00
[
2038-04-01 10:00:00.000000
]
```
whereas I was expecting
```python
2038-04-01 09:00:00+00:00
[
2038-04-01 09:00:00.000000
]
```

Note that it's correct for year 2037 and earlier.

Versions:
- pyarrow: 12.0.0
- python: 3.10.6

### Component(s)

Python

Contributor guide

Open the contributing guide

Research direction

Reproduce the discrepancy with the provided Python example, starting at the pc.assume_timezone entry point and comparing its 2038 result with datetime.astimezone. Trace the timezone conversion path and add or update coverage for America/Boise on 2038-04-01; done means the Arrow result is 09:00 UTC, matching Python.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.