influxdata / influxdata/kapacitor
Kapacitor/Kafka write failures after idle for 9 minutes
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
We are seeing a bug where the kafka event handler doesn't successfully write to kafka after it has been idle for 10 minutes. Basically a few writes fail before the connection to kafka is restarted.
The latest kafka-go, v0.3.4, contains the fix here: https://github.com/segmentio/kafka-go/pull/348 (It is due to a socket idle timeout for kafka producers of 9 minutes.)
I confirmed the fix by adding kafka-go@v0.3.4 to Gopkg.toml like so:
```
[[constraint]]
version = "v0.3.4"
name = "github.com/segmentio/kafka-go"
```
and confirmed that the problem goes away.
Is there any chance you could include segmentio/kafka-go@v0.3.4, (or the latest master,) in the next kapacitor release?
Contributor guide
Research direction
Start by reviewing the Kafka dependency constraint in Gopkg.toml and the Kafka event-handler entry points. Update the dependency to include kafka-go v0.3.4 or the relevant newer version, then verify that writes still succeed after roughly 10 minutes of idle time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100