feast-dev / feast-dev/feast

HybridOnlineStore loses routing_tag after the first FeatureView, so feast apply fails

Open
#6,768 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
1d 21h
Merged PRs (30d)
15

Description

## Expected Behavior

`feast apply` against a `HybridOnlineStore` should route every `FeatureView` to its configured backend, using the tag named by `routing_tag`.

## Current Behavior

It works for the first `FeatureView` and then fails for the second, regardless of how the views are tagged:

```
Applying changes for project mantis
Traceback (most recent call last):
...
File ".../feast/infra/online_stores/hybrid_online_store/hybrid_online_store.py", line 273, in update
raise ValueError(
ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.
```

Note the message says `'tribe'` even though the repo config sets `routing_tag: online_store_dev`. That string is not hardcoded — it is the configured value with `"tribe"` as the fallback:

```python
tag_name = getattr(config.online_store, "routing_tag", "tribe")
```

Getting the fallback means `config.online_store` is no longer the `HybridOnlineStoreConfig` by the time the second view is processed.

### Root cause

Two things combine.

**1. `_prepare_repo_conf` mutates the caller's `RepoConfig` in place.** `rconfig = config` is an alias, not a copy, so both the attribute assignment and the `__dict__` writes land on the caller's object — replacing `online_store` with the selected backend's config. It also injects `type` into `online_store.conf`, which is the user's own dict.

```python
def _prepare_repo_conf(self, config: RepoConfig, online_store_type: str):
rconfig = config # alias, not a copy
for online_store in config.online_store.online_stores:
if online_store.type.split(".")[-1].lower() == online_store_type.lower():
rconfig.online_config = online_store.conf
rconfig.online_config["type"] = online_store.type # mutates caller's conf
data = rconfig.__dict__ # caller's __dict__
data["registry"] = data["registry_config"]
data["offline_store"] = data["offline_config"]
data["online_store"] = data["online_config"] # hybrid config overwritten
return data
```

**2. `update()` rebinds `config` inside the loop over `tables_to_keep`**, so even a non-mutating `_prepare_repo_conf` would feed the next iteration the narrowed config:

```python
for table in tables_to_keep:
tribe = self._get_routing_tag_value(table, config) # config from the previous iteration
if not tribe:
raise ValueError(...)
...
config = RepoConfig(**self._prepare_repo_conf(config, tribe))
```

`teardown()` has the same rebinding inside its loop. `online_write_batch()` and `online_read()` rebind too; those are single-use per call, but they still leave the caller's config mutated through (1).

This makes `HybridOnlineStore` effectively unusable for any repo with more than one `FeatureView`.

## Steps to reproduce

1. Configure a `HybridOnlineStore` with two backends and a custom `routing_tag`:

```yaml
online_store:
type: hybrid
routing_tag: backend
online_stores:
- type: redis
conf: {redis_type: redis, connection_string: "localhost:6379"}
- type: sqlite
conf: {path: "/tmp/feast_hybrid_test.db"}
```

2. Define two `FeatureView`s, one tagged `backend: redis` and one tagged `backend: sqlite`.
3. Run `feast apply`.

The first view is applied; the second raises `ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.`

Equivalent as a test (fails on master, passes with the fix):

```python
HybridOnlineStore().update(
config=repo_config,
tables_to_delete=[],
tables_to_keep=[fv_redis, fv_sqlite],
entities_to_delete=[],
entities_to_keep=[entity],
partial=False,
)
```

### Specifications

- Version: 0.65.0 (also present on `master` as of this writing)
- Platform: Linux / Python 3.12, and reproduced locally on Python 3.11
- Subsystem: `feast.infra.online_stores.hybrid_online_store`

## Possible Solution

- Build the returned mapping from a copy in `_prepare_repo_conf`, and construct a new dict for the backend conf instead of injecting `type` into the caller's.
- Assign the per-backend `RepoConfig` to a local name in `update()` and `teardown()` rather than rebinding the `config` parameter inside the loop.

Happy to open a PR with exactly this plus a unit regression test — I have it working locally.

Contributor guide

Open the contributing guide

Research direction

Start in feast/infra/online_stores/hybrid_online_store/hybrid_online_store.py, reading _prepare_repo_conf and the update and teardown loops. Use the two-FeatureView HybridOnlineStore.update example as a regression test, then verify that each view uses its configured routing tag without mutating the original RepoConfig or backend configuration. Also check the single-call online_write_batch and online_read paths for the same mutation issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.