feast-dev / feast-dev/feast

Support Alibaba Cloud MaxCompute as an offline store

Open
#6,695 0 comments 0 reactions 0 assignees View on GitHub
kind/feature
Dominant language
Python
Stars
7.3k
Forks
1.4k
Avg merge
3d 16h
Merged PRs (30d)
15

Description

**Is your feature request related to a problem? Please describe.**
Feast's built-in offline stores cover BigQuery, Snowflake, Redshift, Spark, Postgres, and file/parquet, but not Alibaba Cloud MaxCompute (ODPS). Users on Alibaba Cloud must export data to parquet or run a separate Spark connector before Feast can read it — adding extra ETL steps, storage cost, and operational overhead.

**Describe the solution you'd like**
A first-class `MaxComputeOfflineStore` plugin, analogous to the existing `BigQueryOfflineStore` / `RedshiftOfflineStore`, so Feast can read directly from a MaxCompute project:

- Implement the standard Feast offline store interface (`OfflineStore`, `RetrievalJob`) using the PyODPS SDK.
- Configure via `feature_store.yaml`: endpoint, project, schema (`odps`), `access_key_id`, `access_key_secret` (AK/SK auth), optional `tunnel_endpoint`.
- Support core operations: point-in-time join for `get_historical_features`, `materialize` / `materialize_incremental`, and offline dataframe push.
- Docs + a minimal end-to-end example.

**Describe alternatives you've considered**
1. Export MaxCompute tables to parquet on OSS, then use Feast's file offline store — adds a redundant export pipeline, duplicates storage, and data is no longer fresh.
2. Spark offline store reading MaxCompute via the Spark connector (what our project currently uses) — works, but requires standing up a dedicated Spark cluster just for Feast, which is operationally heavy for teams already standardized on MaxCompute SQL.
3. Implement as an external contrib plugin outside the Feast repo — reduces discoverability and long-term maintenance.

**Additional context**
- MaxCompute is the dominant data warehouse on Alibaba Cloud, with a large and growing enterprise user base — native Feast support would unlock a significant potential market.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing BigQueryOfflineStore and RedshiftOfflineStore implementations and the OfflineStore and RetrievalJob interfaces, then check how feature_store.yaml configures offline stores. Use the PyODPS SDK as the integration reference; done means supporting the listed historical feature, materialization, and dataframe push operations, with documentation and a minimal end-to-end example.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.