feast-dev / feast-dev/feast

get_online_features silently drops a join key value when it shares a name with an OnDemandFeatureView's RequestSource field

Open Beginner friendly
#6,790 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

## Expected Behavior

When a `FeatureService` combines a regular `FeatureView` and an `OnDemandFeatureView` whose
`RequestSource` happens to declare a field with the same name as one of the `FeatureView`'s
join keys, `get_online_features` should use the provided value for both purposes — as request
data for the ODFV, and as the join key value for the `FeatureView` — the same way
`get_historical_features` already does for equivalent input.

## Current Behavior

`get_online_features` raises a misleading error for the shared-name column, claiming its value
is missing, even though it was genuinely provided in `entity_rows`. `get_historical_features`
given the equivalent `entity_df` succeeds with the same data.

Root cause: `feast/utils.py::_prepare_entities_to_read_from_online_store` classifies each
`entity_rows` key with a **mutually-exclusive** `if`/`elif` chain:

```python
for join_key_or_entity_name, values in entity_proto_values.items():
# Found request data
if join_key_or_entity_name in needed_request_data:
request_data_features[join_key_or_entity_name] = values
elif join_key_or_entity_name in join_keys_set:
# It's a join key
join_key = join_key_or_entity_name
requested_result_row_names.add(join_key)
join_key_values[join_key] = values
elif join_key_or_entity_name in entity_name_to_join_key_map:
...
else:
continue
```

If a name is in `needed_request_data` (because *some* `OnDemandFeatureView` in the query needs
it as a request field), it is classified **only** as request data — even when it is *also* in
`join_keys_set` (because some *other* `FeatureView` in the same query needs it as a join key).
The `elif` branch is never reached for that name, so `join_key_values` never receives it, and
every `FeatureView` expecting that join key later fails with a "missing join key" error, even
though the value was present in `entity_rows` the whole time.

This only affects `get_online_features` — `get_historical_features` does not go through this
function and has no equivalent issue.

## Steps to reproduce

Minimal, self-contained (local file offline store + sqlite online store, no external services):

```python
import os
import shutil
import tempfile
from datetime import datetime, timedelta, timezone

import pandas as pd
from feast import Entity, FeatureService, FeatureStore, FeatureView, Field, FileSource, RequestSource
from feast.on_demand_feature_view import on_demand_feature_view
from feast.types import Float32, String

tmp_dir = tempfile.mkdtemp()
repo_path = tmp_dir + "/repo"
os.makedirs(repo_path, exist_ok=True)

with open(repo_path + "/feature_store.yaml", "w") as f:
f.write(
"project: repro\n"
"provider: local\n"
"registry: registry.db\n"
"online_store:\n"
" type: sqlite\n"
"offline_store:\n"
" type: file\n"
"entity_key_serialization_version: 3\n"
)

data_path = repo_path + "/region_stats.parquet"
now = datetime.now(timezone.utc)
pd.DataFrame({
"region": ["EU", "US"],
"avg_price": [10.0, 20.0],
"event_timestamp": [now - timedelta(days=1), now - timedelta(days=1)],
}).to_parquet(data_path)

region_source = FileSource(path=data_path, timestamp_field="event_timestamp")
region = Entity(name="region", join_keys=["region"])

region_stats_view = FeatureView(
name="region_stats_view",
entities=[region],
schema=[Field(name="avg_price", dtype=Float32)],
source=region_source,
online=True,
)

# RequestSource field is deliberately named "region" too — same name as the join key above.
request_source = RequestSource(
name="context",
schema=[
Field(name="region", dtype=String),
Field(name="multiplier", dtype=Float32),
],
)

@on_demand_feature_view(
sources=[request_source],
schema=[Field(name="region_upper", dtype=String)],
)
def region_odfv(features_df: pd.DataFrame) -> pd.DataFrame:
out = pd.DataFrame()
out["region_upper"] = features_df["region"].str.upper() + "!" + features_df["multiplier"].astype(str)
return out

service = FeatureService(
name="demo_service",
features=[region_stats_view[["avg_price"]], region_odfv],
)

store = FeatureStore(repo_path=repo_path)
store.apply([region, region_source, region_stats_view, region_odfv, service])

entity_rows = [{"region": "EU", "multiplier": 2.0}]

print("=== get_historical_features (works) ===")
entity_df = pd.DataFrame({"region": ["EU"], "multiplier": [2.0], "event_timestamp": [now]})
print(store.get_historical_features(entity_df=entity_df, features=service).to_df())

print("\n=== get_online_features (fails) ===")
print(store.get_online_features(features=service, entity_rows=entity_rows).to_df())

shutil.rmtree(tmp_dir, ignore_errors=True)
```

Output:

```
=== get_historical_features (works) ===
region multiplier event_timestamp avg_price region_upper
0 EU 2.0 2026-08-27 18:24:49.854504+00:00 10.0 EU!2.0

=== get_online_features (fails) ===
KeyError: "Missing join key values for keys: ['region']. No values provided for keys: ['region']. Provided join_key_values: []"
```

### Specifications

- Version: 0.66.0 (latest release on PyPI as of this report)
- Platform: macOS (arm64); also reproduced unmodified inside a Linux container running the
same Feast version
- Subsystem: online feature retrieval (`FeatureStore.get_online_features`)

## Possible Solution

The two membership checks should not be mutually exclusive — a name that is both a needed
request-data field and a join key should be routed to **both** `request_data_features` and
`join_key_values`, e.g.:

```python
for join_key_or_entity_name, values in entity_proto_values.items():
is_request_data = join_key_or_entity_name in needed_request_data
is_join_key = join_key_or_entity_name in join_keys_set
if is_request_data:
request_data_features[join_key_or_entity_name] = values
if is_join_key:
requested_result_row_names.add(join_key_or_entity_name)
join_key_values[join_key_or_entity_name] = values
elif not is_request_data:
if join_key_or_entity_name in entity_name_to_join_key_map:
join_key = entity_name_to_join_key_map[join_key_or_entity_name]
warnings.warn("Using entity name is deprecated. Use join_key instead.")
requested_result_row_names.add(join_key)
join_key_values[join_key] = values
else:
continue
```

I'm not 100% sure this preserves every edge case of the existing (deprecated)
`entity_name_to_join_key_map` fallback branch — flagging the mechanism and a plausible fix
shape, happy to have a maintainer confirm the exact patch.

Contributor guide

Open the contributing guide

Research direction

Start in feast/utils.py at _prepare_entities_to_read_from_online_store and reproduce the issue with the self-contained local offline/SQLite online-store example. Verify that a shared RequestSource and join-key value is retained for both uses and that get_online_features returns the expected features without a missing-key error.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python, sqlite
Domain
backend-api-design, data-engineering, machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.