linkedin / linkedin/Burrow

burrow failure for class-name="kafka"

Open
#444 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

burrow 1.1.0 or master (12e681a3a8a61f84f17677996dc3e6a2b79fac41)
kafka 1.1.0
sasl_plaintext

[logging]
level="debug"

[client-profile.myclient]
kafka-version="1.1.0"
client-id="burrow-myclient"
sasl="saslprofile"

[sasl.saslprofile]
username="xxxxx"
password="xxxxxx"
handshake-first=true

[zookeeper]
servers=[ "kf1:2181" ]
timeout=6
root-path="/burrow"

[cluster.local]
class-name="kafka"
servers=[ "kf1:9092","kf2:9092","kf3:9092" ]
topic-refresh=60
offset-refresh=30
client-profile="myclient"

[consumer.local]
class-name="kafka"
cluster="local"
zookeeper-path="/local"
zookeeper-timeout=30
servers=[ "kf1:9092", "kf2:9092", "kf3:9092" ]

[httpserver.default]
address=":8000"

while starting burrow fails with kafka: client has run out of available brokers to talk to (Is your cluster reachable?)

added debug for samana for broker communication:

[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf2:9092
[sarama] 2018/08/20 16:18:48 SASL authentication successful with broker kf2:9092:4 - [0 0 0 0]
[sarama] 2018/08/20 16:18:48 Connected to broker at kf2:9092 (unregistered)
[sarama] 2018/08/20 16:18:48 client/brokers registered new broker #131368232 at kf3:9092
[sarama] 2018/08/20 16:18:48 client/brokers registered new broker #567700950 at kf2:9092
[sarama] 2018/08/20 16:18:48 client/brokers registered new broker #926183260 at kf1:9092
[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf2:9092
[sarama] 2018/08/20 16:18:48 SASL authentication successful with broker kf2:9092:4 - [0 0 0 0]
[sarama] 2018/08/20 16:18:48 Connected to broker at kf2:9092 (registered as #567700950)
[sarama] 2018/08/20 16:18:48 SASL authentication successful with broker kf1:9092:4 - [0 0 0 0]
[sarama] 2018/08/20 16:18:48 Connected to broker at kf1:9092 (registered as #926183260)
[sarama] 2018/08/20 16:18:48 SASL authentication successful with broker kf3:9092:4 - [0 0 0 0]
[sarama] 2018/08/20 16:18:48 Connected to broker at kf3:9092 (registered as #131368232)
[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf1:9092
[sarama] 2018/08/20 16:18:48 Connected to broker at kf1:9092 (unregistered)
[sarama] 2018/08/20 16:18:48 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:48 Closed connection to broker kf1:9092
[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf3:9092
[sarama] 2018/08/20 16:18:48 Connected to broker at kf3:9092 (unregistered)
[sarama] 2018/08/20 16:18:48 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:48 Closed connection to broker kf3:9092
[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf2:9092
[sarama] 2018/08/20 16:18:48 Connected to broker at kf2:9092 (unregistered)
[sarama] 2018/08/20 16:18:48 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:48 Closed connection to broker kf2:9092
[sarama] 2018/08/20 16:18:48 client/metadata no available broker to send metadata request to
[sarama] 2018/08/20 16:18:48 client/brokers resurrecting 3 dead seed brokers
[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf1:9092
[sarama] 2018/08/20 16:18:48 Connected to broker at kf1:9092 (unregistered)
[sarama] 2018/08/20 16:18:48 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:48 Closed connection to broker kf1:9092
[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf3:9092
[sarama] 2018/08/20 16:18:48 Connected to broker at kf3:9092 (unregistered)
[sarama] 2018/08/20 16:18:48 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:48 Closed connection to broker kf3:9092
[sarama] 2018/08/20 16:18:48 client/metadata fetching metadata for all topics from broker kf2:9092
[sarama] 2018/08/20 16:18:48 Connected to broker at kf2:9092 (unregistered)
[sarama] 2018/08/20 16:18:48 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:48 Closed connection to broker kf2:9092
[sarama] 2018/08/20 16:18:48 client/metadata no available broker to send metadata request to
[sarama] 2018/08/20 16:18:48 client/brokers resurrecting 3 dead seed brokers
[sarama] 2018/08/20 16:18:49 client/metadata fetching metadata for all topics from broker kf1:9092
[sarama] 2018/08/20 16:18:49 Connected to broker at kf1:9092 (unregistered)
[sarama] 2018/08/20 16:18:49 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:49 Closed connection to broker kf1:9092
[sarama] 2018/08/20 16:18:49 client/metadata fetching metadata for all topics from broker kf3:9092
[sarama] 2018/08/20 16:18:49 Connected to broker at kf3:9092 (unregistered)
[sarama] 2018/08/20 16:18:49 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:49 Closed connection to broker kf3:9092
[sarama] 2018/08/20 16:18:49 client/metadata fetching metadata for all topics from broker kf2:9092
[sarama] 2018/08/20 16:18:49 Connected to broker at kf2:9092 (unregistered)
[sarama] 2018/08/20 16:18:49 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:49 Closed connection to broker kf2:9092
[sarama] 2018/08/20 16:18:49 client/metadata no available broker to send metadata request to
[sarama] 2018/08/20 16:18:49 client/brokers resurrecting 3 dead seed brokers
[sarama] 2018/08/20 16:18:49 client/metadata fetching metadata for all topics from broker kf1:9092
[sarama] 2018/08/20 16:18:49 Connected to broker at kf1:9092 (unregistered)
[sarama] 2018/08/20 16:18:49 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:49 Closed connection to broker kf1:9092
[sarama] 2018/08/20 16:18:49 client/metadata fetching metadata for all topics from broker kf3:9092
[sarama] 2018/08/20 16:18:49 Connected to broker at kf3:9092 (unregistered)
[sarama] 2018/08/20 16:18:49 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:49 Closed connection to broker kf3:9092
[sarama] 2018/08/20 16:18:49 client/metadata fetching metadata for all topics from broker kf2:9092
[sarama] 2018/08/20 16:18:49 Connected to broker at kf2:9092 (unregistered)
[sarama] 2018/08/20 16:18:49 client/metadata got error from broker while fetching metadata: EOF
[sarama] 2018/08/20 16:18:49 Closed connection to broker kf2:9092
[sarama] 2018/08/20 16:18:49 client/metadata no available broker to send metadata request to
[sarama] 2018/08/20 16:18:49 client/brokers resurrecting 3 dead seed brokers
{"level":"error","ts":1534781929.483589,"msg":"failed to start client","type":"module","coordinator":"consumer","class":"kafka","name":"local","error":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)"}

for some reason only first connection to the brokers for collecting metadata was successful, next attempts fails. nothing interesting in kafka logs.
can somebody help me with this?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce startup with the provided Burrow, Kafka, SASL, cluster, and consumer configuration, then inspect the Sarama debug output around the successful first metadata connection and subsequent EOF failures. No source file or test is named; done means identifying why metadata requests lose all brokers and confirming that Burrow starts successfully with the affected setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.