apache / apache/pinot

RealtimeProvisioningHelper does not account for deepStore

Open
#7,994 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

The RealtimeProvisioningHelper assumes that you need all the segments in memory in the realtime host, this isn't the case if you are pushing completed segments to deepStore. It would be nice if we could tell the realtime provisioning helper that deep store is enabled so that it can account for this.

Contributor guide

Open the contributing guide

Research direction

Start by locating RealtimeProvisioningHelper and tracing how it calculates realtime-host memory for segments. Identify how deepStore is configured and where completed segments leave memory; done means the helper can be told that deepStore is enabled and adjusts its accounting accordingly, with relevant tests covering both modes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.