create FeatureStore from configs in cloud storage (s3, gcp, etc)
- Dominant language
- Python
- Stars
- 7.3k
- Forks
- 1.4k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 15
Description
**Is your feature request related to a problem? Please describe.**
The cli commands use `ceate_feature_store` ([code](https://github.com/robhowley/feast/blob/ac6529c380bc623a15054350c9936b69617b4bee/sdk/python/feast/repo_operations.py#L334-L336)) to create the feature store from the `click.Context`. It is required that the `repo_path` be local and contain the `feature_store.yaml` file ([code](https://github.com/robhowley/feast/blob/ac6529c380bc623a15054350c9936b69617b4bee/sdk/python/feast/repo_operations.py#L380-L387)).
It would be helpful and simpler for deployments if it was possible to provide cloud storage URIs for `repo_path`. The `feature_store.yaml` file will vary based upon deployment environment, e.g. local vs staging vs prod, so decoupling it from the code artifact would make ci/cd pipelines cleaner. Currently, if you wanted to run feast out of a container the image would have to include the `feature_store.yaml` in it. You'd have a dockerfile similar to
```
FROM feastdev/feature-server:0.35.0
ENV DEPLOY_ENV
COPY ./feature_store/feature_store.{DEPLOY_ENV}.yaml ./app/feature_store/feature_store.yaml
WORKDIR /app
```
which is a bit of busy work, an extra layer of indirection, and a slower build time.
**Describe the solution you'd like**
Would like to be able to set cloud URIs in `FS_YAML_FILE`. We'd then have nicely containerized deployments that could do things like ...
```
docker run \
-e FS_YAML_FILE=s3://{env}-feature-store/feature_store.yaml \
feastdev/feature-server:0.35.0 feast ui
```
The code and configuration are now separate. The actual loading of the file can leverage a package like `cloudpathlib` so that we can work in the familiar `pathlib.Path` interface.
**Describe alternatives you've considered**
Could also make `repo_path` a cli parameter, but that would make the "start server" commands a bit bloated. It is nice having the parameters focus on server issues as opposed to `feature_store.yaml` issues.
Contributor guide
Assessment
This issue has not been assessed yet.