apache / apache/pinot

Make time boundary offset configurable

Open
#6,256 5 comments 0 reactions 1 assignee Claimed by @chenboat View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Currently for Pinot hybrid tables, the timeboundary offset (used to decide the split of querying either offline table or realtime table) is hardcoded to be 1 day of daily offline tables. This hardcoded value may not work well in many environments. In particular, different companies have different batch data refreshment rate: some can be well under 1 day.

It is much better to leave this field to be a configurable parameter for Pinot broker.

https://github.com/apache/incubator-pinot/blob/0f4ee6814a5fa103f5b88d4df75b0968dc7ce641/pinot-broker/src/main/java/org/apache/pinot/broker/routing/timeboundary/TimeBoundaryManager.java#L159

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.