kafka delay metric not reporting when count drops to zero
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Bug Description
What happened:
The kafkaDelayCount metric stops being emitted when Kafka lag drops to zero because the metric call in KafkaOffsetGen.getNextOffsetRanges() is inside a checkpoint validation conditional block.
What you expected:
The metric should always be emitted, reporting 0 when caught up rather than disappearing/flatlining from monitoring dashboards.
Steps to reproduce:
Start DeltaStreamer with Kafka source and metrics enabled
Wait for consumer to catch up (latestOffsets == committedOffsets)
Observe kafkaDelayCount metric is no longer emitted
### Environment
**Hudi version:**
**Query engine:** (Spark/Flink/Trino etc)
**Relevant configs:**
### Logs and Stack Trace
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with KafkaOffsetGen.getNextOffsetRanges(), where the issue says the kafkaDelayCount metric call is inside checkpoint validation. Reproduce the DeltaStreamer scenario with Kafka metrics enabled and the consumer caught up, then verify that the metric remains emitted with a value of 0 when latestOffsets equals committedOffsets.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- data-engineering, stream-processing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100