ray-project/raydp

View on GitHub

RayDP provides simple APIs for running Spark on Ray and integrating Spark with AI libraries.

Stars: 377
Forks: 85
Open beginner issues: 0
Indexed issues: 40
Avg merge: 3d 20h
Merged PRs (30d): 2

Dominant language: Scala
License: Apache-2.0
Last GitHub push: Sep 11, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

40 open issues indexed

RayDP crashes with no exception after adding lance-spark-3.5_2.12.jar Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

ray-project/raydp#469 · 1 comment · 1 reaction · Opened Mar 27, 2026
Could not modify bound address for processes created for spark master Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#434 · Opened Aug 4, 2025
raydp.init_spark support sparkSession parameter Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#433 · 3 comments · Opened Jul 17, 2025
Removing usage of private/unstable Ray APIs Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

ray-project/raydp#430 · 1 comment · Opened May 30, 2025
Exception while creating a udf Open

Difficulty 3/5 1-2 days Newbie friendliness 25/100

ray-project/raydp#429 · 1 comment · Opened Apr 9, 2025
CNCF TAG Runtime Discussion/Presentation Open

Difficulty 5/5 Over a week Newbie friendliness 15/100

ray-project/raydp#418 · Opened Oct 25, 2024
add support of init_spark from existing SparkSession? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#416 · 4 reactions · Opened Sep 24, 2024
Slack Group for the commiters & maintainers Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ray-project/raydp#415 · 4 comments · Opened Sep 18, 2024
Can you pass spark.driver.extraClassPath directly into raydp.init()? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#414 · Opened Sep 4, 2024
RayDP support for spark-connect Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ray-project/raydp#412 · 4 comments · 5 reactions · Opened Jun 27, 2024
incompatible with spark 3.4.3 and 3.5.0 Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

ray-project/raydp#409 · 4 comments · Opened Jun 4, 2024
how to init raydp resource Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

ray-project/raydp#408 · 1 comment · Opened May 23, 2024
raydp.init_spark fails Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#404 · 1 comment · Opened Apr 17, 2024
install raydp==1.6.0 with pip rasied the "More than one SparkShimProvider found" error Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

ray-project/raydp#402 · Opened Mar 5, 2024
Is it possible for raydp to utilize a Spark session created by Apache Livy? Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ray-project/raydp#394 · 1 comment · Opened Dec 14, 2023
[stream] does raydp support spark stream data? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

ray-project/raydp#393 · 1 comment · Opened Dec 12, 2023
Getting Java gateway process exited before sending its port number during init_spark() Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

ray-project/raydp#383 · 11 comments · Opened Oct 6, 2023
Runtime env_vars cannot be passed down to executors Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#380 · 2 comments · 1 reaction · Opened Sep 21, 2023
When submit a script another process start locally in the head node Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#372 · 1 comment · Opened Aug 9, 2023
Support ray 2.3.1 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#371 · 3 comments · Opened Aug 8, 2023
raydp start occur null error Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

ray-project/raydp#357 · 1 comment · Opened Jun 30, 2023
Issues with running on Ray Client Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

ray-project/raydp#352 · 3 comments · Opened Jun 5, 2023
raydp.init_spark fails Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

ray-project/raydp#350 · 3 comments · Opened Jun 2, 2023
Threads hang on getting from object store when calling `to_spark` Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#348 · Opened May 30, 2023
Data ownership transfer tests fail in Ray 2.4.0 Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

ray-project/raydp#343 · Opened May 12, 2023
Performance benchmark on RayDP v.s. Spark Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ray-project/raydp#340 · 3 comments · Opened May 5, 2023
Support/Validate Ray 2.4.0 Release Open

Difficulty 4/5 3-5 days Newbie friendliness 20/100

ray-project/raydp#339 · Opened Apr 27, 2023
Running RayDP on GPU machine stuck at GCS readiness Open

Difficulty 4/5 3-5 days Newbie friendliness 32/100

ray-project/raydp#336 · 1 comment · Opened Apr 19, 2023
fix the workaround of working with ray 2.3.[0-1] Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

ray-project/raydp#334 · Opened Apr 18, 2023
Fix security issue of protobuf < 3.19.5 Open

Difficulty 2/5 1-3 hours Newbie friendliness 35/100

ray-project/raydp#332 · Opened Apr 18, 2023
[Feature] Write your feature request here!🚀 Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ray-project/raydp#331 · 3 comments · Opened Apr 17, 2023
Building Docker image to run Spark on Ray with RAPIDS Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ray-project/raydp#330 · 2 comments · Opened Apr 9, 2023
rename ray JVM log file to avoid being monitored and polled Open

Difficulty 2/5 1-3 hours Newbie friendliness 45/100

ray-project/raydp#327 · Opened Apr 4, 2023
Failed validating 'additionalProperties' in schema when using Python 3.8 for docker example Open

Difficulty 4/5 3-5 days Newbie friendliness 30/100

ray-project/raydp#305 · 1 comment · Opened Feb 14, 2023
请问本项目是否有计划支持Flink Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ray-project/raydp#291 · 1 comment · Opened Dec 1, 2022
Implement shuffle service based on ray object store Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

ray-project/raydp#276 · 1 comment · 1 reaction · Opened Oct 4, 2022
Using RayDp with SparkNLP gives an exception when trying to view the spark dataframe columns Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#217 · 10 comments · Opened Mar 2, 2022
Integrate Spark Streaming with Ray/Ray Datasets Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

ray-project/raydp#213 · 3 comments · Opened Feb 11, 2022
conda-forge package? Open

Difficulty 5/5 Over a week Newbie friendliness 20/100

ray-project/raydp#124 · 3 comments · Opened Apr 4, 2021
Having worker nodes specific for Spark jobs Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

ray-project/raydp#119 · 2 comments · Opened Apr 1, 2021

ray-project/raydp

Get new issues in your inbox