[Data] Support to read from/ write to hologres in ray
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43.9k
- Forks
- 8.1k
- PR merge metrics
- PR metrics pending
Description
Description
Hologres is a real-time interactive analytics engine that is compatible with PostgreSQL, developed by Alibaba Cloud(website). The default read/write interface cannot distribute the operation in parallel for hologres, so new APIs with optimizations specific to hologres can be helpful for hologres users, especially working on large scale datasets.
Use case
With new APIs, users can read from/ write to hologres in an efficient way in ray, typical use cases are like
ds = ray.data.read_hologres(
schema_name="public",
table_name="my_table",
connection_uri="postgresql://username:password@host:port/database",
...
)
ds.write_hologres(
schema_name="public",
table_name="my_table",
connection_uri=hologres_connection_uri,
...
)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing Ray Data's existing read and write APIs and the proposed read_hologres and write_hologres entry points. Clarify the Hologres-specific parallelization and connection requirements, then define the API behavior and tests needed to show efficient reads and writes for the example use cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- data-engineering, databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100