feast-dev / feast-dev/feast

`RemoteOnlineStore.online_write_batch` hardcodes timestamp column names (`created`/`event_timestamp`), breaking remote `write_to_online_store` for feature views with a custom `created_timestamp_column`

Open Beginner friendly
#6,595 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

## Expected Behavior

Remote `write_to_online_store` should round-trip regardless of the feature view's
timestamp column names. Writing works for a local provider (the full dataframe is
converted in-process using the source's configured column names) and should behave the
same when the online store is remote (`online_store: { type: remote }` / the feature
server `/write-to-online-store` endpoint).

## Current Behavior

Writing to the online store through the remote path fails on the server with:

```
KeyError: 'Field "created_at" does not exist in schema'
```

whenever the feature view's batch source sets `created_timestamp_column` (or
`timestamp_field`) to anything other than the literals `"created"` /
`"event_timestamp"`.

Root cause — `RemoteOnlineStore.online_write_batch` serializes the timestamps into
**hardcoded** column names when building the request body:

```python
# sdk/python/feast/infra/online_stores/remote.py (online_write_batch)
columnar_data["event_timestamp"].append(_to_naive_utc(event_ts).isoformat())
columnar_data["created"].append(
_to_naive_utc(created_ts).isoformat() if created_ts else None
)
req_body = {"feature_view_name": table.name, "df": columnar_data, ...}
post_remote_online_write(config=config, req_body=req_body)
```

The server rebuilds the dataframe verbatim and runs the standard conversion:

```python
# sdk/python/feast/feature_server.py
df = pd.DataFrame(request.df) # columns: entities, features, event_timestamp, created
store.write_to_online_store(..., df=df)
# -> _prep_rows_to_write_for_ingestion -> _convert_arrow_to_proto
```

but `_convert_arrow_fv_to_proto` (in `utils.py`) resolves the created-timestamp column
by the **feature-view-configured name**, not the hardcoded `"created"`:

```python
if feature_view.batch_source.created_timestamp_column:
created_timestamps = [
... table.column(
feature_view.batch_source.created_timestamp_column # e.g. "created_at"
) ...
]
```

So the client emits the column as `created` and the server looks it up as `created_at`
→ `KeyError`. The timestamp **value is not lost** — it is on the wire under `created`;
it is simply **labeled with a name the server does not look up**. The client write and
the server-side conversion disagree on the timestamp column names.

Server-side traceback:

```
File ".../feast/feature_server.py", in write_to_online_store
File ".../feast/feature_store.py", in write_to_online_store
File ".../feast/infra/passthrough_provider.py", in _prep_rows_to_write_for_ingestion
File ".../feast/utils.py", in _convert_arrow_fv_to_proto
KeyError: 'Field "created_at" does not exist in schema'
```

## Steps to reproduce

1. Define a feature view whose batch source uses non-default timestamp column names,
e.g. `PostgreSQLSource(timestamp_field="event_timestamp",
created_timestamp_column="created_at", ...)`.
2. Point a client at a remote online store: `online_store: { type: remote, path:
}` (or call the feature server `/write-to-online-store`).
3. `store.write_to_online_store(fv_name, df=df)` where `df` includes the `created_at`
column. The client-side conversion succeeds; the server returns HTTP 500 with the
`KeyError` above. The same write against a local provider succeeds.

### Specifications

- Version: 0.64.0 (client and feature server)
- Platform: Linux, Python 3.13
- Subsystem: remote online store (`feast/infra/online_stores/remote.py`) /
Python feature server `/write-to-online-store` (`feast/feature_server.py`) /
`_convert_arrow_fv_to_proto` (`feast/utils.py`)

## Possible Solution

In `RemoteOnlineStore.online_write_batch`, label the timestamp columns using the
feature view's configured fields instead of hardcoded literals:

```python
event_col = table.batch_source.timestamp_field or "event_timestamp"
created_col = table.batch_source.created_timestamp_column or "created"
columnar_data[event_col].append(...)
if created_col:
columnar_data[created_col].append(...)
```

Alternatively, have the server map the well-known `event_timestamp` / `created`
request columns onto the FV's configured names before conversion. Fixing the client to
emit the configured names is the smaller change and keeps the request self-describing.

Contributor guide

Open the contributing guide

Research direction

Start in sdk/python/feast/infra/online_stores/remote.py at RemoteOnlineStore.online_write_batch, then inspect the /write-to-online-store path in sdk/python/feast/feature_server.py and timestamp handling in sdk/python/feast/utils.py. Reproduce the remote write with custom timestamp names and verify that feature views using both custom and default names round-trip successfully without the KeyError.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
api, backend, data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.