Join in Python is very slow
Open
- Dominant language
- Java
- Stars
- 274
- Forks
- 140
- Avg merge
- 5d 16h
- Merged PRs (30d)
- 4
Description
It is sending every data point one by one to Python for the udf to run and then sent back to Java
Contributor guide
Research direction
The issue names no file or test; start by tracing the Python Join execution path and the Java-to-Python data exchange. Measure the per-point transfer described in the report, then verify that the completed change reduces exchange overhead while preserving correct join results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- data-engineering, distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100