Stale realtimeingestiondelayms metric for paused topic in multi-topic tables
Open
ingestion
real-time
troubleshooting
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
When auto offset reset is enabled, Pinot automatically pauses ingestion on the backfill topic when backfilling is done. But the ingestion delay keeps increasing until server restart.
Contributor guide
Research direction
Start by locating the realtimeingestiondelayms metric and tracing how it is updated for paused backfill topics in multi-topic tables. Reproduce the auto offset reset scenario, then verify that the metric no longer increases after ingestion pauses and remains correct without a server restart.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100