influxdata / influxdata/kapacitor

Kapacitor/Kafka write failures after idle for 9 minutes

Open
#2,260 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

We are seeing a bug where the kafka event handler doesn't successfully write to kafka after it has been idle for 10 minutes. Basically a few writes fail before the connection to kafka is restarted.

The latest kafka-go, v0.3.4, contains the fix here: https://github.com/segmentio/kafka-go/pull/348 (It is due to a socket idle timeout for kafka producers of 9 minutes.)

I confirmed the fix by adding kafka-go@v0.3.4 to Gopkg.toml like so:
```
[[constraint]]
version = "v0.3.4"
name = "github.com/segmentio/kafka-go"
```
and confirmed that the problem goes away.

Is there any chance you could include segmentio/kafka-go@v0.3.4, (or the latest master,) in the next kapacitor release?

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the Kafka dependency constraint in Gopkg.toml and the Kafka event-handler entry points. Update the dependency to include kafka-go v0.3.4 or the relevant newer version, then verify that writes still succeed after roughly 10 minutes of idle time.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
stream-processing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.