Confluent Cloud Setup

Hi guys,

I've been struggling with the burrow setup for quite some time for kafka cluster on confluent cloud. Burrow has been really valuable piece of software for the inhouse deployed kafka clusters but since my team moved to the confluent cloud I've been struggling to set it up correctly.

Here's the config:

[zookeeper]
servers=[ "172.17.0.4:2181" ]
timeout=6
root-path="/burrow"

[logging]
level="debug"

[cluster.stg]
class-name="kafka"
servers=[ "*****.us-east1.gcp.confluent.cloud:9092" ]
client-profile="myclient"
topic-refresh=60
offset-refresh=30

[consumer.stg]
class-name="kafka"
cluster="stg"
client-profile="myclient"
servers=[ "*****.us-east1.gcp.confluent.cloud:9092" ]
group-allowlist=""

[client-profile.myclient]
kafka-version="0.11.0"
client-id="burrow-myclient"
sasl="mysaslprofile"

[sasl.mysaslprofile]
username="*********"
password="*********"
handshake-first=false

[httpserver.default]
address=":8000"

I was trying all possible config combinations and the one above is the one that got me to pass the SASL auth but it's still failing as client metadata gets EOF error. Please see below:

Reading configuration from /etc/burrow
{"level":"info","ts":1597737468.3371112,"msg":"Started Burrow"}
{"level":"info","ts":1597737468.3375235,"msg":"configuring","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1597737468.3385446,"msg":"configuring","type":"coordinator","name":"storage"}
{"level":"info","ts":1597737468.3386433,"msg":"configuring","type":"module","coordinator":"storage","class":"inmemory","name":"default"}
{"level":"info","ts":1597737468.3388054,"msg":"configuring","type":"coordinator","name":"evaluator"}
{"level":"info","ts":1597737468.3388784,"msg":"configuring","type":"module","coordinator":"evaluator","class":"caching","name":"default"}
{"level":"info","ts":1597737468.3389807,"msg":"configuring","type":"coordinator","name":"httpserver"}
{"level":"info","ts":1597737468.3399022,"msg":"configuring","type":"coordinator","name":"notifier"}
{"level":"info","ts":1597737468.3400364,"msg":"configuring","type":"coordinator","name":"cluster"}
{"level":"info","ts":1597737468.3400853,"msg":"configuring","type":"module","coordinator":"cluster","class":"kafka","name":"oc_prod"}
{"level":"info","ts":1597737468.3452342,"msg":"configuring","type":"coordinator","name":"consumer"}
{"level":"info","ts":1597737468.3453486,"msg":"configuring","type":"module","coordinator":"consumer","class":"kafka","name":"prod"}
{"level":"info","ts":1597737468.3459487,"msg":"starting","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1597737468.3561957,"msg":"Connected to 172.17.0.4:2181","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1597737468.4135559,"msg":"authenticated: id=72138833186127922, timeout=6000","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1597737468.4136565,"msg":"re-submitting `0` credentials after reconnect","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1597737468.4246602,"msg":"starting","type":"coordinator","name":"storage"}
{"level":"info","ts":1597737468.42482,"msg":"starting","type":"module","coordinator":"storage","class":"inmemory","name":"default"}
{"level":"info","ts":1597737468.4251058,"msg":"starting","type":"coordinator","name":"evaluator"}
{"level":"info","ts":1597737468.4251423,"msg":"starting","type":"module","coordinator":"evaluator","class":"caching","name":"default"}
{"level":"info","ts":1597737468.425219,"msg":"starting","type":"coordinator","name":"httpserver"}
{"level":"info","ts":1597737468.4268248,"msg":"started listener","type":"coordinator","name":"httpserver","listener":"[::]:8000"}
{"level":"info","ts":1597737468.427205,"msg":"starting","type":"coordinator","name":"notifier"}
{"level":"info","ts":1597737468.4276655,"msg":"starting","type":"coordinator","name":"cluster"}
{"level":"info","ts":1597737468.4277368,"msg":"starting","type":"module","coordinator":"cluster","class":"kafka","name":"oc_prod"}
{"level":"debug","ts":1597737468.4280841,"msg":"Initializing new client","name":"sarama"}
{"level":"debug","ts":1597737468.4307697,"msg":"client/metadata fetching metadata for all topics from broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"info","ts":1597737468.5562336,"msg":"starting evaluations","type":"coordinator","name":"notifier"}
{"level":"debug","ts":1597737468.7360644,"msg":"SASL authentication successful with broker ******.us-east1.gcp.confluent.cloud:9092:4 - [21 3 3 0]","name":"sarama"}
{"level":"debug","ts":1597737468.7361805,"msg":"Connected to broker at ******.us-east1.gcp.confluent.cloud:9092 (unregistered)","name":"sarama"}
{"level":"debug","ts":1597737468.738494,"msg":"client/metadata got error from broker -1 while fetching metadata: unexpected EOF","name":"sarama"}
{"level":"debug","ts":1597737468.7386537,"msg":"Closed connection to broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"debug","ts":1597737468.7387395,"msg":"client/metadata no available broker to send metadata request to","name":"sarama"}
{"level":"debug","ts":1597737468.7387767,"msg":"client/brokers resurrecting 1 dead seed brokers","name":"sarama"}
{"level":"debug","ts":1597737468.7388234,"msg":"client/metadata retrying after 250ms... (3 attempts remaining)","name":"sarama"}
{"level":"debug","ts":1597737468.989536,"msg":"client/metadata fetching metadata for all topics from broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"debug","ts":1597737469.2316642,"msg":"SASL authentication successful with broker ******.us-east1.gcp.confluent.cloud:9092:4 - [21 3 3 0]","name":"sarama"}
{"level":"debug","ts":1597737469.231808,"msg":"Connected to broker at ******.us-east1.gcp.confluent.cloud:9092 (unregistered)","name":"sarama"}
{"level":"debug","ts":1597737469.2322161,"msg":"client/metadata got error from broker -1 while fetching metadata: unexpected EOF","name":"sarama"}
{"level":"debug","ts":1597737469.232594,"msg":"Closed connection to broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"debug","ts":1597737469.2328062,"msg":"client/metadata no available broker to send metadata request to","name":"sarama"}
{"level":"debug","ts":1597737469.2328613,"msg":"client/brokers resurrecting 1 dead seed brokers","name":"sarama"}
{"level":"debug","ts":1597737469.2329073,"msg":"client/metadata retrying after 250ms... (2 attempts remaining)","name":"sarama"}
{"level":"debug","ts":1597737469.483665,"msg":"client/metadata fetching metadata for all topics from broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"debug","ts":1597737469.7172434,"msg":"SASL authentication successful with broker ******.us-east1.gcp.confluent.cloud:9092:4 - [21 3 3 0]","name":"sarama"}
{"level":"debug","ts":1597737469.7200181,"msg":"Connected to broker at ******.us-east1.gcp.confluent.cloud:9092 (unregistered)","name":"sarama"}
{"level":"debug","ts":1597737469.7209947,"msg":"client/metadata got error from broker -1 while fetching metadata: unexpected EOF","name":"sarama"}
{"level":"debug","ts":1597737469.7211635,"msg":"Closed connection to broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"debug","ts":1597737469.721262,"msg":"client/metadata no available broker to send metadata request to","name":"sarama"}
{"level":"debug","ts":1597737469.7216485,"msg":"client/brokers resurrecting 1 dead seed brokers","name":"sarama"}
{"level":"debug","ts":1597737469.7218363,"msg":"client/metadata retrying after 250ms... (1 attempts remaining)","name":"sarama"}
{"level":"debug","ts":1597737469.972495,"msg":"client/metadata fetching metadata for all topics from broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"debug","ts":1597737470.2147756,"msg":"SASL authentication successful with broker ******.us-east1.gcp.confluent.cloud:9092:4 - [21 3 3 0]","name":"sarama"}
{"level":"debug","ts":1597737470.2148404,"msg":"Connected to broker at ******.us-east1.gcp.confluent.cloud:9092 (unregistered)","name":"sarama"}
{"level":"debug","ts":1597737470.2154193,"msg":"client/metadata got error from broker -1 while fetching metadata: unexpected EOF","name":"sarama"}
{"level":"debug","ts":1597737470.215738,"msg":"Closed connection to broker ******.us-east1.gcp.confluent.cloud:9092","name":"sarama"}
{"level":"debug","ts":1597737470.2158542,"msg":"client/metadata no available broker to send metadata request to","name":"sarama"}
{"level":"debug","ts":1597737470.2159607,"msg":"client/brokers resurrecting 1 dead seed brokers","name":"sarama"}
{"level":"debug","ts":1597737470.216271,"msg":"Closing Client","name":"sarama"}
{"level":"error","ts":1597737470.2164073,"msg":"failed to start client","type":"module","coordinator":"cluster","class":"kafka","name":"oc_prod","error":"kafka: client has run out of available brokers to talk to (Is your cluster reachable?)"}
{"level":"info","ts":1597737470.2164896,"msg":"stopping","type":"coordinator","name":"notifier"}
{"level":"info","ts":1597737470.2165515,"msg":"shutdown","type":"coordinator","name":"httpserver"}
{"level":"info","ts":1597737470.2169995,"msg":"stopping","type":"coordinator","name":"evaluator"}
{"level":"info","ts":1597737470.217062,"msg":"stopping","type":"module","coordinator":"evaluator","class":"caching","name":"default"}
{"level":"info","ts":1597737470.217476,"msg":"stopping","type":"coordinator","name":"storage"}
{"level":"info","ts":1597737470.2175245,"msg":"stopping","type":"module","coordinator":"storage","class":"inmemory","name":"default"}
{"level":"info","ts":1597737470.2175949,"msg":"stopping","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1597737470.22605,"msg":"recv loop terminated: err=EOF","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1597737470.2261844,"msg":"send loop terminated: err=<nil>","type":"coordinator","name":"zookeeper"}
Burrow failed at August 18, 2020 at 7:57am (UTC)

I was wondering if any of you had any experience in setting up burrow against managed kafka cluster in the confluent cloud. It seems to me this is something with either confluent cloud design or my cluster config but it's really strange that this example is not documented anywhere. I guess managed kafka service will be getting more adoption so documenting this use case will be valuable for the wider community.

结合 Burrow 的 Kafka 客户端和客户端配置文件文档，审查 issue 中显示的 Confluent Cloud 配置。从 Sarama 的元数据 EOF 日志入手，确定该设置是否受支持，或是否需要记录配置指南。完成的标准是提供一个清晰、可复现的 Confluent Cloud 设置，或明确说明该限制。