linkedin / linkedin/Burrow

Cannot find storm consumers and kafka consumers

Open
#779 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

Hi,

We were trying to use Burrow tool for monitoring lags in our storm and Kafka consumers.
However we do not see any storm consumers returned via API. Also not all Kafka consumers are returned.Can someone help to figure out the issue.

**Below is the config we are using:**

[general]
pidfile="burrow.pid"
stdout-logfile="burrow.out"
access-control-allow-origin="*"

[logging]
filename="logs/burrow.log"
level="info"
maxsize=100
maxbackups=30
maxage=10
use-localtime=false
use-compression=true

[zookeeper]
servers=["localhost:2181" ]
timeout=6 #expiration timeout for zookeeper sessions.
root-path="/kafka" #full path to zookeeper node that burrow will be allowed to write into.

[client-profile.myclient]
client-id="burrow-myclient"
kafka-version="0.10.1"

[cluster.local]
class-name="kafka"
servers=[ "localhost:6667" ]
client-profile="myclient" #profile name defined as in client profile subheading.
topic-refresh=10
offset-refresh=10

[consumer.local]
class-name="kafka"
cluster="local" # sub heading name defined in above cluster configuration
servers=[ "localhost:6667" ]
client-profile="myclient"

[consumer.local_zk]
class-name="kafka_zk"
cluster="local"
servers=["localhost:2181" ]
zookeeper-timeout=30
zookeeper-path=""
zookeeper-offsets=true

[httpserver.mylistener]
address=":8080"

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the missing-consumer responses using the supplied Burrow configuration, focusing on the consumer.local and consumer.local_zk sections and the configured Kafka and ZooKeeper endpoints. Compare the API results with the Storm and Kafka consumers that are expected; done means identifying the cause of the omissions and documenting or fixing it.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
backend, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.