get_historical_features raises TypeError on a zero-row entity_df (Dask/file offline store)
- Dominant language
- Python
- Stars
- 7.3k
- Forks
- 1.4k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 15
Description
## Expected Behavior
`get_historical_features` with an `entity_df` that happens to have zero rows should
return an empty result with the right columns. A zero-row entity frame is a normal
degenerate case in batch scoring — the upstream query simply matched nothing that run.
## Current Behavior
It raises an opaque pandas error from deep inside the Dask offline store:
```
TypeError: Invalid comparison between dtype=datetime64[ns] and DatetimeArray
```
Traceback tail:
```
File "sdk/python/feast/infra/offline_stores/offline_store.py", line 178, in to_arrow
features_table = self._to_arrow_internal(timeout=timeout)
File "sdk/python/feast/infra/offline_stores/dask.py", line 103, in _to_arrow_internal
df = self.evaluation_function().compute()
File "sdk/python/feast/infra/offline_stores/dask.py", line 317, in evaluate_historical_retrieval
df_to_join = _filter_ttl(
File "sdk/python/feast/infra/offline_stores/dask.py", line 1206, in _filter_ttl
df_to_join = df_to_join.persist()
```
## Steps to reproduce
Identical construction, only the row count differs, against a `file` offline store:
```python
def mk(n):
return pd.DataFrame({
"driver_id": [1001] * n,
"event_timestamp": pd.to_datetime([datetime(2026, 2, 1)] * n, utc=True),
})
fs.get_historical_features(entity_df=mk(1), features=["driver_stats:conv_rate"]).to_df()
# OK, shape=(1, 3)
fs.get_historical_features(entity_df=mk(0), features=["driver_stats:conv_rate"]).to_df()
# TypeError: Invalid comparison between dtype=datetime64[ns] and DatetimeArray
```
### Specifications
- Version: `master` @ `5ad5592390febfca60c9d88edf7daccbdd156fd6`
- Platform: Linux x86_64, Python 3.11.15, dask 2026.8.0
- Subsystem: offline store (Dask / file)
## Possible Solution
`_normalize_timestamp` (`sdk/python/feast/infra/offline_stores/dask.py:1139`) makes
timestamp columns tz-aware with a row-wise `apply`:
```python
df_to_join[timestamp_field] = df_to_join[timestamp_field].apply(
lambda x: x if x.tzinfo else x.replace(tzinfo=timezone.utc),
meta=(timestamp_field, "datetime64[ns, UTC]"),
)
```
`meta` *declares* a tz-aware result, but with zero rows the lambda never runs, so the
computed partition stays `datetime64[ns]`. Declared and actual dtypes then diverge, and
the tz-naive vs tz-aware comparison in `_filter_ttl`
(`sdk/python/feast/infra/offline_stores/dask.py:1183`) raises.
Replacing the row-wise `apply` with a vectorized, empty-safe conversion fixes it and is
faster on non-empty frames too: pick `dt.tz_localize("UTC")` for a tz-naive column and
`dt.tz_convert("UTC")` otherwise, based on the column's declared dtype. That yields the
correct dtype even when the partition is empty.
A regression test covering a zero-row `entity_df` through
`get_historical_features(...).to_df()` on the Dask store would pin this down; the
current suite only exercises non-empty entity frames, which is why CI stays green.
Happy to send a PR for this.
Contributor guide
Research direction
Start in sdk/python/feast/infra/offline_stores/dask.py at _normalize_timestamp, _filter_ttl, and evaluate_historical_retrieval; reproduce the failure with an empty entity_df through get_historical_features(...).to_df(). Add a regression test for the Dask/file offline store and verify that both empty and non-empty inputs complete with the expected columns and dtypes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100