influxdata / influxdata/telegraf
Cancel gracefully when connecting to kafka broker
- Dominant language
- Go
- Stars
- 17.8k
- Forks
- 5.8k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 161
Description
If the user sends a signal to telegraf while kafka_consumer is connecting to the broker (before sarama.NewConsumerGroup() has returned), the signal will be delayed until connection is made or timed out. This means sigint to shut down telegraf, or sighup to reload config are ignored while connecting to the broker.
Since #12111, telegraf allows the kafka connection to be deferred until after startup and to retry for a very long time in case the broker is down. This means the time spent connecting to the broker can be much larger and signals can be blocked indefinitely.
It doesn't look like there is a way to cancel sarama.NewConsumerGroup(). We need to find out if there is a different way to connect that has a way to cancel, or work with upstream to add one.
Contributor guide
Research direction
Start with the kafka_consumer connection path around sarama.NewConsumerGroup() and review whether Sarama provides a cancellable connection method. Determine how deferred startup and retry behavior interact with SIGINT and SIGHUP handling. Done means connection attempts can be interrupted so shutdown and configuration reload signals are not blocked indefinitely.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100