Optimize /tables/{tableName}/size endpoint to avoid expensive server fan-out on every call
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
## Problem:
The controller endpoint `GET /tables/{tableName}/size` performs a synchronous HTTP fan-out to every server hosting segments for the requested table. Each server computes sizes by recursively walking segment directories on disk via `FileUtils.sizeOfDirectory()`. The overall latency is bounded by the slowest server to respond, making this endpoint slow for tables with many segments spread across many servers.
Meanwhile, SegmentStatusChecker — a periodic task running on every controller — already calls `TableSizeReader.getTableSizeDetails()` for every table that controller owns, performing the exact same server fan-out. However, it discards the computed TableSizeDetails result after emitting metrics, so the work is wasted from the REST API's perspective.
For clients that don't need real-time size data, paying the full fan-out cost on every API call is unnecessary.
## Proposed Solution:
1. Add an in-memory cache to TableSizeReader: Store the TableSizeDetails result after each computation. Since SegmentStatusChecker already calls `getTableSizeDetails()` periodically for its owned tables, the cache is automatically kept warm with staleness bounded by controller.statuschecker.frequencyInSeconds.
2. Add a mode=cache query parameter to the REST endpoint: When mode=cache is specified, the endpoint returns the cached result instead of triggering a live fan-out. Default behavior (no mode parameter) remains unchanged for backward compatibility.
3. Redirect to the owning controller: Tables are partitioned across controllers via the lead controller resource — each controller only owns ~1/N of the tables and only runs SegmentStatusChecker for those tables. When a mode=cache request lands on a controller that doesn't own the requested table, it proxy the request to the owning controller, which has the warm cache. Owning controller resolution uses existing infrastructure (LeadControllerUtils.getPartitionIdForTable() + Helix external view).
## Benefits:
- Eliminates redundant server fan-out for clients that can tolerate slightly stale data (default to refresh every 5 min)
- No new threads, timers, or background tasks — piggybacks on existing SegmentStatusChecker infrastructure
- Fully backward-compatible (opt-in via query parameter)
- Redirect pattern ensures the request always reaches the controller with the warm cache, regardless of which controller the VIP routes to
Contributor guide
Research direction
Start at the GET /tables/{tableName}/size endpoint and trace TableSizeReader.getTableSizeDetails() and SegmentStatusChecker. Verify how LeadControllerUtils.getPartitionIdForTable() and the Helix external view identify the owning controller. Done means cache mode avoids a live fan-out when possible, redirects non-owning requests correctly, and leaves the default behavior unchanged.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend, distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100