apache / apache/pinot

Stale realtimeingestiondelayms metric for paused topic in multi-topic tables

Open
#18,065 0 comments 0 reactions 0 assignees View on GitHub
ingestion real-time troubleshooting
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

When auto offset reset is enabled, Pinot automatically pauses ingestion on the backfill topic when backfilling is done. But the ingestion delay keeps increasing until server restart.

Contributor guide

Open the contributing guide

Research direction

Start by locating the realtimeingestiondelayms metric and tracing how it is updated for paused backfill topics in multi-topic tables. Reproduce the auto offset reset scenario, then verify that the metric no longer increases after ingestion pauses and remains correct without a server restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.