HybridOnlineStore loses routing_tag after the first FeatureView, so feast apply fails
- Dominant language
- Python
- Stars
- 7.3k
- Forks
- 1.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 15
Description
## Expected Behavior
`feast apply` against a `HybridOnlineStore` should route every `FeatureView` to its configured backend, using the tag named by `routing_tag`.
## Current Behavior
It works for the first `FeatureView` and then fails for the second, regardless of how the views are tagged:
```
Applying changes for project mantis
Traceback (most recent call last):
...
File ".../feast/infra/online_stores/hybrid_online_store/hybrid_online_store.py", line 273, in update
raise ValueError(
ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.
```
Note the message says `'tribe'` even though the repo config sets `routing_tag: online_store_dev`. That string is not hardcoded — it is the configured value with `"tribe"` as the fallback:
```python
tag_name = getattr(config.online_store, "routing_tag", "tribe")
```
Getting the fallback means `config.online_store` is no longer the `HybridOnlineStoreConfig` by the time the second view is processed.
### Root cause
Two things combine.
**1. `_prepare_repo_conf` mutates the caller's `RepoConfig` in place.** `rconfig = config` is an alias, not a copy, so both the attribute assignment and the `__dict__` writes land on the caller's object — replacing `online_store` with the selected backend's config. It also injects `type` into `online_store.conf`, which is the user's own dict.
```python
def _prepare_repo_conf(self, config: RepoConfig, online_store_type: str):
rconfig = config # alias, not a copy
for online_store in config.online_store.online_stores:
if online_store.type.split(".")[-1].lower() == online_store_type.lower():
rconfig.online_config = online_store.conf
rconfig.online_config["type"] = online_store.type # mutates caller's conf
data = rconfig.__dict__ # caller's __dict__
data["registry"] = data["registry_config"]
data["offline_store"] = data["offline_config"]
data["online_store"] = data["online_config"] # hybrid config overwritten
return data
```
**2. `update()` rebinds `config` inside the loop over `tables_to_keep`**, so even a non-mutating `_prepare_repo_conf` would feed the next iteration the narrowed config:
```python
for table in tables_to_keep:
tribe = self._get_routing_tag_value(table, config) # config from the previous iteration
if not tribe:
raise ValueError(...)
...
config = RepoConfig(**self._prepare_repo_conf(config, tribe))
```
`teardown()` has the same rebinding inside its loop. `online_write_batch()` and `online_read()` rebind too; those are single-use per call, but they still leave the caller's config mutated through (1).
This makes `HybridOnlineStore` effectively unusable for any repo with more than one `FeatureView`.
## Steps to reproduce
1. Configure a `HybridOnlineStore` with two backends and a custom `routing_tag`:
```yaml
online_store:
type: hybrid
routing_tag: backend
online_stores:
- type: redis
conf: {redis_type: redis, connection_string: "localhost:6379"}
- type: sqlite
conf: {path: "/tmp/feast_hybrid_test.db"}
```
2. Define two `FeatureView`s, one tagged `backend: redis` and one tagged `backend: sqlite`.
3. Run `feast apply`.
The first view is applied; the second raises `ValueError: FeatureView must have a 'tribe' tag to use HybridOnlineStore.`
Equivalent as a test (fails on master, passes with the fix):
```python
HybridOnlineStore().update(
config=repo_config,
tables_to_delete=[],
tables_to_keep=[fv_redis, fv_sqlite],
entities_to_delete=[],
entities_to_keep=[entity],
partial=False,
)
```
### Specifications
- Version: 0.65.0 (also present on `master` as of this writing)
- Platform: Linux / Python 3.12, and reproduced locally on Python 3.11
- Subsystem: `feast.infra.online_stores.hybrid_online_store`
## Possible Solution
- Build the returned mapping from a copy in `_prepare_repo_conf`, and construct a new dict for the backend conf instead of injecting `type` into the caller's.
- Assign the per-backend `RepoConfig` to a local name in `update()` and `teardown()` rather than rebinding the `config` parameter inside the loop.
Happy to open a PR with exactly this plus a unit regression test — I have it working locally.
Contributor guide
Research direction
Start in feast/infra/online_stores/hybrid_online_store/hybrid_online_store.py, reading _prepare_repo_conf and the update and teardown loops. Use the two-FeatureView HybridOnlineStore.update example as a regression test, then verify that each view uses its configured routing tag without mutating the original RepoConfig or backend configuration. Also check the single-call online_write_batch and online_read paths for the same mutation issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100