googleapis / googleapis/python-aiplatform

`pyspark` connector to read & write from/to `Vertex AI Feature Store`

Open
#2,554 0 comments 0 reactions 0 assignees View on GitHub
api: vertex-ai
Dominant language
Python
Stars
905
Forks
465
Avg merge
1d 13h
Merged PRs (30d)
44

Description

- Currently, one can only use ai platform sdk or pandas to do CRUD operations on Vertex AI feature store.
- It would be nice, if there's a way to utilize `pyspark` to do the same so that one can scale their ETL for these operations for large datasets

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points; start by reviewing the existing AI Platform SDK and pandas CRUD paths for Vertex AI Feature Store and how a PySpark ETL connector would fit. Done means a defined way to read and write Feature Store data through PySpark at large-dataset scale.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python, spark
Domain
cloud, data-engineering, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.