feast-dev / feast-dev/feast

Allow serving online features while a FeatureView is `MATERIALIZING` (config to relax the lifecycle serving gate)

Open
#6,780 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
3d 16h
Merged PRs (30d)
15

Description

## Summary
Since **v0.64.0**, `get_online_features` refuses to serve any FeatureView whose lifecycle state is
not `AVAILABLE_ONLINE`/`STATE_UNSPECIFIED`. Because `store.materialize()` transitions a FV to
`MATERIALIZING` in the (shared) registry for the duration of its run, **online serving fails for that
FV while it is being materialized**. For the canonical setup — a long-running `feast serve` plus a
periodic (incremental) materialization job sharing one registry — this turns routine materialization
into recurring online-serving outages. Requesting a config option to keep serving the last-materialized
values during `MATERIALIZING`.

## Current behavior
`sdk/python/feast/utils.py` (`_get_online_features`) enforces an unconditional gate:

```python
raise ValueError(
f"Feature view '{name}' is in state '{fv.state.name}' "
f"and cannot serve features. Only AVAILABLE_ONLINE feature views can serve."
)
```

`FeatureStore.materialize()` sets the FV to `MATERIALIZING` and commits it to the registry before
writing (`feature_view.state = FeatureViewState.MATERIALIZING; self.registry.apply_feature_view(..., commit=True)`).
A separate feature server reading the same registry (subject to `cache_ttl_seconds`) then observes
`MATERIALIZING` and rejects requests. Since a `get_online_features` call spans all requested FVs, a
**single** materializing FV fails the **entire** request (including any feature service that references it).

## Steps to reproduce
1. `feast apply`, then materialize once so the FV is `AVAILABLE_ONLINE`.
2. Start `feast serve`; confirm `/get-online-features` works.
3. Run `feast materialize` for that FV (or set its state to `MATERIALIZING`).
4. Call `/get-online-features` for the same FV during the run →
`ValueError: Feature view '' is in state 'MATERIALIZING' and cannot serve features.`

## Impact
- Continuous online serving + periodic incremental materialization is the standard online pattern;
with one shared registry the serving tier unavoidably observes `MATERIALIZING`.
- Materialization is exactly when the *previous* values are still valid and should keep being served;
instead reads hard-fail.
- Blast radius is the whole request / feature service, not just the one FV.

## When it was introduced
The gate was added in **PR #6401** (“Add enabled/disabled toggle for feature views”), first released in
**v0.64.0**, and is present unchanged in **v0.65.0**, **v0.66.0**, and `master`. It is unconditional —
there is no config to opt out. Versions ≤ v0.63.x do not have it.

## Proposed solution
Add a config flag (e.g. under `online_store` or top-level in `feature_store.yaml`) such as:

```yaml
serve_during_materialization: true # default false to preserve current behavior
```

When enabled, `_get_online_features` treats `MATERIALIZING` as servable (serves the last-materialized
values) instead of raising. `GENERATED` / never-materialized FVs can still be gated. Alternatively,
scope the gate so `MATERIALIZING` is always servable (only truly-unavailable states are rejected),
since a FV that is re-materializing already has prior data online.

## Environment
- Feast: 0.65.0 (also confirmed in 0.64.0, 0.66.0, master)
- Online store: Redis · Offline store: Athena · Registry: file (S3), shared between `feast serve`
and a scheduled `feast materialize`

Contributor guide

Open the contributing guide

Research direction

Start in sdk/python/feast/utils.py at _get_online_features and trace FeatureStore.materialize, which commits MATERIALIZING to the shared registry. Review feature_store.yaml and the feast serve/materialize entry points to determine where the opt-in setting belongs. Done means the configured behavior serves last-materialized values during MATERIALIZING while GENERATED or never-materialized views remain gated, with the existing default preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, redis
Domain
backend, data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.