linkedin / linkedin/Burrow

burrow 1.0 reports only one consumer while two consumers are active

Open
#319 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted
Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

I am migrating to burrow 1.0 our configuration and I've noticed that I have only one consumer available:

```
[kkrzyzaniak@kafka01:~]$ curl --silent http://localhost:8000/v3/kafka/central/consumer
{
"error": false,
"message": "consumer list returned",
"consumers": [
"logstash-turbo"
],
"request": {
"url": "/v3/kafka/central/consumer",
"host": "kafka01"
}
}
```

While there are two active consumers:

```
[kkrzyzaniak@kafka01:~]$ /opt/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).

logstash
logstash-turbo
```

My config:

```
[general]
pidfile="/run/burrow/burrow.pid"
stdout-logfile="/var/log/burrow/burrow.out"
access-control-allow-origin=""

[logging]
filename="/var/log/burrow/burrow.log"
level="info"
maxsize=100
maxbackups=30
maxage=10
use-localtime=false
use-compression=true

[zookeeper]
servers=[ "zk01:2181", "zk02:2181", "zk03:2181" ]
timeout=6
root-path="/burrow"

[client-profile.test]
client-id="burrow-test"
kafka-version="0.10.0"

[cluster.central]
class-name="kafka"
servers=[ "kafka01:9092", "kafka02:9092", "kafka03:9092" ]
client-profile="test"
topic-refresh=120
offset-refresh=30

[consumer.central]
class-name="kafka"
cluster="central"
servers=[ "kafka01:9092", "kafka02:9092", "kafka03:9092" ]
client-profile="test"
group-blacklist=""
group-whitelist=""

[httpserver.default]
address=":8000"

[storage.default]
class-name="inmemory"
workers=20
intervals=15
expire-group=604800
min-distance=1
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the /v3/kafka/central/consumer endpoint and compare its consumer list with the kafka-consumer-groups.sh --list output shown in the report. Trace how the central consumer configuration discovers groups, then verify that both logstash and logstash-turbo appear in the endpoint response.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
backend, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.