ray-project / ray-project/raydp
[Feature] Write your feature request here!🚀
Open
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 377
- Forks
- 85
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 2
Description
Here we will maintain a list of requested features, to make it easier to track.
- 1. Add custom resource requirements for RayDP executor actors. This has been implemented for RayDP master actors thanks to @pang-wu . Related issue #119
- 2. Release conda package. Related issue #124
- 3. Support Ray Dataset Pipelining with Spark Dataframe
- 4. Support streaming. Spark Streaming/Flink/Kafka? Need more use cases. Related issue #213 #291
- 5. Provide a guide for using RayDP in Ray on Kubernetes. Related issue #305
- 6. Integration with Spark native SQL engine Gluten
Feel free to write yours below! Also welcome to share your use case or scenario of RayDP, this will help us better understand your feature request, and how to further optimize performance, thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by choosing one unchecked feature from the list and reviewing its related issue where provided, such as #119, #124, #213, #291, or #305. The issue names no files, tests, or single entry point; completion would require defining the selected feature's scope and updating the request list when that feature is implemented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kafka, kubernetes, scala, spark
- Domain
- cloud, data-engineering, distributed-systems, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100