apache / apache/pinot

Optimize /tables/{tableName}/size endpoint to avoid expensive server fan-out on every call

Open
#18,077 2 comments 0 reactions 0 assignees View on GitHub
PEP-Request
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

## Problem:

The controller endpoint `GET /tables/{tableName}/size` performs a synchronous HTTP fan-out to every server hosting segments for the requested table. Each server computes sizes by recursively walking segment directories on disk via `FileUtils.sizeOfDirectory()`. The overall latency is bounded by the slowest server to respond, making this endpoint slow for tables with many segments spread across many servers.

Meanwhile, SegmentStatusChecker — a periodic task running on every controller — already calls `TableSizeReader.getTableSizeDetails()` for every table that controller owns, performing the exact same server fan-out. However, it discards the computed TableSizeDetails result after emitting metrics, so the work is wasted from the REST API's perspective.

For clients that don't need real-time size data, paying the full fan-out cost on every API call is unnecessary.

## Proposed Solution:

1. Add an in-memory cache to TableSizeReader: Store the TableSizeDetails result after each computation. Since SegmentStatusChecker already calls `getTableSizeDetails()` periodically for its owned tables, the cache is automatically kept warm with staleness bounded by controller.statuschecker.frequencyInSeconds.
2. Add a mode=cache query parameter to the REST endpoint: When mode=cache is specified, the endpoint returns the cached result instead of triggering a live fan-out. Default behavior (no mode parameter) remains unchanged for backward compatibility.
3. Redirect to the owning controller: Tables are partitioned across controllers via the lead controller resource — each controller only owns ~1/N of the tables and only runs SegmentStatusChecker for those tables. When a mode=cache request lands on a controller that doesn't own the requested table, it proxy the request to the owning controller, which has the warm cache. Owning controller resolution uses existing infrastructure (LeadControllerUtils.getPartitionIdForTable() + Helix external view).

## Benefits:
- Eliminates redundant server fan-out for clients that can tolerate slightly stale data (default to refresh every 5 min)
- No new threads, timers, or background tasks — piggybacks on existing SegmentStatusChecker infrastructure
- Fully backward-compatible (opt-in via query parameter)
- Redirect pattern ensures the request always reaches the controller with the warm cache, regardless of which controller the VIP routes to

Contributor guide

Open the contributing guide

Research direction

Start at the GET /tables/{tableName}/size endpoint and trace TableSizeReader.getTableSizeDetails() and SegmentStatusChecker. Verify how LeadControllerUtils.getPartitionIdForTable() and the Helix external view identify the owning controller. Done means cache mode avoids a live fan-out when possible, redirects non-owning requests correctly, and leaves the default behavior unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend, distributed-systems, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.