apache / apache/pinot

Provide a standalone jar with Pinot's partition functions

Open
#8,887 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Often, users need to ensure that the upstream job creating pinot segments uses the same partitioning function that Pinot table expects. It will be much easier if we can provide these functions as a separate jar (without a lot of pinot dependencies) so that users can use it in their spark or other ingestion jobs.

For more context, see thread - https://apache-pinot.slack.com/archives/C011C9JHN7R/p1654790723474439

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the issue context and the linked Slack thread to identify which Pinot partition functions upstream jobs must share. Determine the standalone JAR's dependency boundary and verify that it can be used by Spark or other ingestion jobs without most Pinot dependencies; completion means the requested artifact is available for those consumers.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.