tamnd / tamnd/firepanda

to_datetime on a column that is neither text nor whole numbers

Open
#353 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Mojo
Stars
1
Forks
0
PR merge metrics
PR metrics pending

Description

`firepanda.to_datetime` has two readings, one for text and one for whole numbers, and a column that is neither falls into the numeric one and is refused there. pandas has a third answer for one of those types and a different exception class for another, and both differences are measured below against pandas 3.0.5.

## What was measured

| input | pandas | firepanda |
|---|---|---|
| `[1.5, 2.5]` | `datetime64[ns]` | `InvalidArgumentError`, which is a `ValueError` |
| `[True, False]` | `TypeError: is not convertible to datetime` | `InvalidArgumentError`, which is a `ValueError` |
| `[[1], [2]]` | `TypeError` | `DTypeError`, which is a `TypeError`, so this one already agrees |
| `[{"a": 1}]` | `TypeError` | `DTypeError`, so this one already agrees too |
| `["2026-01-01", 5]` | `ValueError` | `DTypeError`, raised by the Series constructor before `to_datetime` sees it |

## The float

pandas reads a float column as a count from the epoch in whatever `unit` says, the same as it reads an integer one, and it keeps the fractional part. `pd.to_datetime([1.5], unit="s")` is one and a half seconds after the epoch. firepanda refuses it with a message about needing whole numbers.

This is the one of the five that is a missing reading rather than a wrong exception. It is not hard: the fraction is scaled by the same factor the whole part is and then rounded, and the rounding rule is the thing to check against pandas rather than assume, since a half unit is exactly the value where the two obvious rules disagree.

## The bool

pandas raises `TypeError` here, on the same distinction Python draws everywhere else: the type was wrong rather than the value. firepanda raises a `ValueError`, because the kernel refuses it as an argument problem and `PySeries.to_datetime` retags everything that comes out of the kernel as `VALUE`.

A caller writing `except TypeError` around a `to_datetime` on data of unknown provenance catches it in pandas and does not catch it here. The fix is in the Python layer rather than the kernel, since that is where the column's own type is already known before the call is made, and it is the same place `_refuse` and `_held_at` already look at what they were handed.

## The mixed column

`["2026-01-01", 5]` never reaches `to_datetime` at all. `firepanda.Series` refuses to build a column that mixes integers and text, so the error comes from the constructor and says so. pandas builds an object column and then fails to read row two against the format it worked out from row one, which is a `ValueError`.

This one is arguably not a bug. The two libraries disagree about whether such a column can exist, and firepanda's answer is the Arrow one. It is recorded here because a caller reading the exception class sees the difference, and because the three cases should be decided together rather than one at a time.

## Suggested order

The float reading first, since it is a real gap and the other two are about which class the failure wears. Then the bool, in the Python layer. The mixed column last, and possibly as a decision to leave it alone.

Contributor guide

Open the contributing guide

Research direction

Start at firepanda.to_datetime and the PySeries.to_datetime entry point, then read the Python-layer handling in _refuse and _held_at. Compare float and bool cases with pandas, including fractional-unit rounding, and decide whether the mixed-column difference remains intentional. Done means float columns match pandas and bool failures expose TypeError consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.