apache / apache/pinot

Framework to certify Pinot release (functional + performance)

Open
#5,490 13 comments 6 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

With the Pinot community growing, as well as Pinot evolving for the better, we anticipate a ton of features that would go into performance critical parts of the code (e.g. query execution, RT segment generation, etc). Given that Pinot's forte is its blazingly fast speed, we would like to ensure that the rapid feature development does not compromise on its performance and functionality. We need a framework using which any PR can be evaluated for performance/functionality. Some ideas:

Query Performance:
1. Create a few synthetic data sets with different characteristics that represent existing Pinot deployments across the community. We can probably enhance the existing data generator tool to server these requirements.
2. Generate a query set for a given data set to ensure desired level of selectivity.
3. Framework that runs a desired workload (qps) on these data sets with and without the changes from a PR, and reports the performance changes.

Realtime:
1. Consumption rate
2. Segment generation time

Ability to check on a variety of metrics, for example:
1. Latency p95/p99
2. Throughput
3. GC

https://docs.google.com/document/d/1QweQObrwGrcMR47dKzg1ZQpQkj4udHttkNm5kcMVBQ8/edit?usp=sharing

Contributor guide

Open the contributing guide

Research direction

Start with the linked design document and inspect Pinot's existing data generator tool, which the issue identifies as a possible foundation. Define the framework scope from the listed query and realtime workloads and metrics, then confirm that it can compare PR changes for functionality, latency, throughput, GC, consumption rate, and segment generation time.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, performance, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.