apache / apache/pinot

Improve/Add completeness and freshness metrics for Pinot

Open
#4,007 4 comments 0 reactions 0 assignees View on GitHub
metrics observability
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 55m
Merged PRs (30d)
182

Description

This issue tracks the need to expose better metrics in Pinot to expose completeness and freshness in the query path. Today, a lot of individual metrics are available for tracking, but its typically hard to say which queries were actually affected when issues do happen. Allowing tracking/metadata to be sent in query response helps with this. This issue tracks efforts being made for this.
NOTE: the tracking is on a best effort basis - we would like to track the most common possibilities (without tying ourselves to specific individual scenarios) without compromising the execution time of the query and continuing to keep the design/implementation as simple/intuitive as possible.

Details:
======
Completeness:
-------------------
Currently, we set a "partial-response" metadata flag when a broker is unable to get a connection to the server or response from a server within configured timeout. We would like to expand this to cover:
(a) cases where servers don't have the segments the broker is requesting.
(b) broker's routing table is incomplete - for ex, when all the replicas of a given segment are ofline or in error state.

Freshness:
---------------
In the current scope, we limit the freshness of the data to realtime tables. We would like to measure how "recent" the data available for querying in realtime is.

More details for each of these will follow.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing "partial-response" metadata through Pinot's broker query path and reviewing the individual metrics already exposed. Determine how missing segments, incomplete routing, and realtime data age should be represented without compromising query execution time; the issue does not yet identify specific files, tests, or final acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems, observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.