Provide a standalone jar with Pinot's partition functions
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Often, users need to ensure that the upstream job creating pinot segments uses the same partitioning function that Pinot table expects. It will be much easier if we can provide these functions as a separate jar (without a lot of pinot dependencies) so that users can use it in their spark or other ingestion jobs.
For more context, see thread - https://apache-pinot.slack.com/archives/C011C9JHN7R/p1654790723474439
Contributor guide
Research direction
Start by reviewing the issue context and the linked Slack thread to identify which Pinot partition functions upstream jobs must share. Determine the standalone JAR's dependency boundary and verify that it can be used by Spark or other ingestion jobs without most Pinot dependencies; completion means the requested artifact is available for those consumers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100