Abstract away BlockingQueue.offer()/poll() which notify after some period of inability to proceed
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 2d 58m
- Merged PRs (30d)
- 233
Description
There are several places in Druid code which implement some sort of untimed `BlockingQueue.take()`, but they want to notify or log about inability to make progress, so they use `BlockingQueue.offer()`:
https://github.com/apache/incubator-druid/blob/fafbc4a80e0bba0c1fbb5066b9f0ce37ffcab929/extensions-contrib/ambari-metrics-emitter/src/main/java/org/apache/druid/emitter/ambari/metrics/AmbariMetricsEmitter.java#L115-L127
https://github.com/apache/incubator-druid/blob/fafbc4a80e0bba0c1fbb5066b9f0ce37ffcab929/examples/src/main/java/org/apache/druid/examples/twitter/TwitterSpritzerFirehoseFactory.java#L165-L168
https://github.com/apache/incubator-druid/blob/fafbc4a80e0bba0c1fbb5066b9f0ce37ffcab929/extensions-core/kinesis-indexing-service/src/main/java/org/apache/druid/indexing/kinesis/KinesisRecordSupplier.java#L176-L179
https://github.com/apache/incubator-druid/blob/fafbc4a80e0bba0c1fbb5066b9f0ce37ffcab929/server/src/main/java/org/apache/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java#L376-L383
https://github.com/apache/incubator-druid/blob/fafbc4a80e0bba0c1fbb5066b9f0ce37ffcab929/extensions-contrib/graphite-emitter/src/main/java/org/apache/druid/emitter/graphite/GraphiteEmitter.java#L109-L121
They all use different strategies and patterns. It might be reasonable to extract this pattern as some sort of utility, or e. g. specify a common "timeout" in druid after which such events are logged, or a configuration might be added to choose between logging and emitting such events as alerts.
Contributor guide
Research direction
Start by comparing the queue handling in AmbariMetricsEmitter.java, TwitterSpritzerFirehoseFactory.java, KinesisRecordSupplier.java, EventReceiverFirehoseFactory.java, and GraphiteEmitter.java. Identify the common progress-notification behavior and decide whether a utility, shared timeout, or configurable alerting approach is intended; done requires a clearly defined design applied consistently across the listed entry points.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100