feast-dev / feast-dev/feast

On-demand feature view UDFs lose their module globals when rebuilt from the registry, so serving fails with NameError

Open
#6,815 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
3d 16h
Merged PRs (30d)
15

Description

## Expected Behavior

An OnDemandFeatureView whose UDF calls a helper (or reads a constant/import) defined
at its own module level should keep working when the view is loaded back from the
registry — e.g. in a feature server, or any process that did not define the view itself.

## Current Behavior

The UDF is rebuilt from its stored source text only, so every name it resolves from
its defining module is missing, and the transformation fails at call time:

```
NameError: name 'scaled' is not defined
```

`PandasTransformation.from_proto` / `PythonTransformation.from_proto` prefer
`rehydrate_udf_from_source` over the stored dill body. That function `exec`s **only the
function's own source** into a fresh namespace seeded with just `pd`/`pandas`/`np`/`numpy`
(`sdk/python/feast/transformation/udf_rehydrate.py:102`), so a module-level helper,
constant, or an import under any other alias is not present.

The `exec` itself still succeeds, because a function body's free variables are only
resolved when the function is *called*. So `resolve_udf`
(`sdk/python/feast/transformation/udf_rehydrate.py:160`) treats rehydration as
successful and **never falls back to the dill body** — even when that body would have
worked. The failure surfaces later, during retrieval.

## Steps to reproduce

Apply an ODFV from a script/notebook, so dill serializes the UDF by value and captures
its globals:

```python
SCALE = 100.0

def scaled(x):
return x * SCALE

@on_demand_feature_view(
sources=[stats, req],
schema=[Field(name="conv_rate_scaled", dtype=Float64)],
)
def scaled_rate(inputs: pd.DataFrame) -> pd.DataFrame:
df = pd.DataFrame()
df["conv_rate_scaled"] = scaled(inputs["conv_rate"]) + inputs["bonus"]
return df

fs.apply([driver, src, stats, req, scaled_rate])
fs.materialize_incremental(datetime(2026, 6, 1))
```

Then read it back in a separate process that only has the registry (what a feature
server does):

```python
fs = FeatureStore(repo_path=".")
fs.get_online_features(
features=["driver_stats:conv_rate", "scaled_rate:conv_rate_scaled"],
entity_rows=[{"driver_id": 1001, "bonus": 1.0}],
).to_dict()
# NameError: name 'scaled' is not defined
```

Monkeypatching `from_proto` back to the previous dill-only behaviour, in the same
process and against the same registry, returns the correct result:

```python
@classmethod
def _old_from_proto(cls, p):
return PandasTransformation(udf=dill.loads(p.body), udf_string=p.body_text)

PandasTransformation.from_proto = _old_from_proto
# -> conv_rate_scaled=[51.0]
```

So the registry demonstrably holds a working body that is being ignored.

Both transformation modes are affected — a bare proto round-trip is enough to show it:

```python
t = PythonTransformation(udf=py_udf, udf_string=src) # py_udf calls helper()
rt = PythonTransformation.from_proto(t.to_proto())

t.udf({"a": [1, 2]}) # {'out': [3, 6]}
rt.udf({"a": [1, 2]}) # NameError: name 'helper' is not defined
```

### Specifications

- Version: `master` @ `5ad5592390febfca60c9d88edf7daccbdd156fd6` (source-first rehydration added in #6655)
- Platform: Linux x86_64, Python 3.11.15
- Subsystem: on-demand feature views / transformation serialization

## Possible Solution

Make the "is this rehydrated callable usable?" check non-vacuous before accepting it.
Walk the compiled function's `co_names` (recursively through nested code objects) and
confirm each is resolvable in the exec namespace or in builtins; if any name is missing,
return `None` so `resolve_udf` falls back to the dill body as it did before.

A `NameError`-catching wrapper that retries via dill on first call would also work, but
validating up front keeps the failure out of the serving path entirely.

Worth noting this is distinct from #5620, which covers the dill-by-reference case where
the defining module genuinely is not importable. The problem here is that source-first
rehydration is chosen even when the stored dill body would succeed, so a setup that
previously served correctly now raises.

I'm happy to put up a PR for this if the validate-then-fall-back approach looks right.

Contributor guide

Open the contributing guide

Research direction

Start with sdk/python/feast/transformation/udf_rehydrate.py, especially rehydrate_udf_from_source and resolve_udf, then inspect PandasTransformation.from_proto and PythonTransformation.from_proto. Reproduce the bare proto round-trip with a helper-based UDF; done means module globals no longer cause serving-time NameError and a usable stored dill body is used when source rehydration is incomplete.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.