feast-dev / feast-dev/feast

OnDemandFeatureView silently skips transformation if BatchSource defines column with same name

Open
#5,534 1 comment 0 reactions 0 assignees View on GitHub
kind/bug priority/p2
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

## **Summary**

When an `OnDemandFeatureView` defines a transformed feature (e.g. `revenue_plus_one`), and the upstream `BatchSource` (usually through a `FeatureView`) already defines a column with the **same name**, Feast **silently skips** applying the ODFV transformation. This can have negative impact if the existing column contains only `None` values/ other incorrect data**.

This behavior leads to **silent inconsistencies** in the offline path and contradicts the intent of using ODFVs for transformation logic.

### **Current Behavior**

* ODFVs apply their transformation in the **offline path only if** the output column **does not exist** in the `BatchSource`.
* If the `BatchSource` includes a column with the same name (even with bad/missing values), Feast assumes it is the final value and skips transformation.
* This affects reading features from offline store with `get_historical_features()`.
* This might happen incidentally, because the `FeatureView` used as input to the ODFV points to a `BatchSource` that might include unexpected or unknown columns.

---

### **Steps to Reproduce**

1. Create a `FeatureView` from a `BatchSource` that includes a column `"revenue_plus_one"` filled with `None` or incorrect values.
2. Define an `OnDemandFeatureView` that expects to compute `"revenue_plus_one"` from base features (e.g. `revenue + 1`).
3. Use the ODFV in a historical feature retrieval.

```python
features = [
"customer_stats:revenue",
"revenue_plus_one:revenue_plus_one", # from ODFV
]

# BatchSource has revenue_plus_one = None
df = store.get_historical_features(entity_df=..., features=features).to_df()
print(df["revenue_plus_one"]) # Contains None instead of expected transformed values
```

---

### **Expected Behavior**

* The `OnDemandFeatureView` (ODFV) should **not unknowingly use a column** from an upstream `FeatureView`’s `BatchSource` just because the column name matches the ODFV’s output.
* If a column with the same name exists in the `BatchSource`, Feast should:

* Either still apply the transformation from the ODFV,
* Or explicitly verify that the existing column is valid (non-null, correct type),
* Or *warn the user about the name collision and skipped transformation.
* The behavior should be consistent and explainable, so users know when transformations were applied and when they were not.

---

### **Root Cause**

Feast uses the output column name as a selector. If the column exists in the `BatchSource`, Feast treats it as final — regardless of whether the transformation logic would produce different or more complete results.

This causes subtle and difficult-to-detect **Training-Serving Skew**, especially in pipelines where `FeatureViews` and `BatchSources` are reused.

---

### **Proposed Solutions**

To avoid incorrect behavior when Feast skips On-Demand Feature View (ODFV) transformations due to existing columns in the batch source, I propose two possible solutions:

#### 1. Marker Field to Track Transformation

Add an internal marker or metadata tag that indicates whether a feature value has already been transformed.

This can be used to:

Ensure the transformation is only skipped if it really already happened.

In the simplest form, this marker just records if the transformation was ever applied.

Longer-term, this could help build a full understanding of how each feature value was created.

#### 2. Use the Compute Engine DAG to Manage Transformations Explicitly

The new DAG-based Compute Engine could explicitly control when and how each transformation is applied.

Instead of relying on column names, the system would always run transformations defined in the DAG, unless they are marked as already applied (via the marker field above).

---

### **Additional Context**

This issue is especially confusing because the `FeatureView` may define a `BatchSource`, which silently passes through unknown columns. These columns **won’t be used** when the `FeatureView` is queried directly, but **will be used** if an `ODFV` is layered on top with the same output name.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.