linkedin / linkedin/Burrow

Burrow reporting topics that were deleted from Kafka

Open
#458 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

**Issue we are seeing:**
Our instance of Burrow is based on commit f3659d121a36fb9ebfd2519c024a132260323bb0
We deleted N number of topics from Kafka
Requests to /v3/kafka/{cluster-name}/topic still return those deleted topics.

Besides setting topic-refresh, is there another configuration we are missing or have mis-configured causing burrow to retain deleted topics?

We have the following configuration

`[general]
pidfile="burrow.pid"
stdout-logfile="burrow.out"

[logging]
filename="logs/burrow.log"
level="info"
maxsize=100
maxbackups=30
maxage=30
use-localtime=false
use-compression=true

[zookeeper]
servers=["10.10.X.X:2181", "10.10.Y.Y:2181", "10.10.Z.Z:2181"]
timeout=6
root-path="/burrow"

[client-profile.client]
kafka-version="0.10.2.0"
client-id="burrow-client"

[cluster.prod-kafka]
class-name="kafka"
client-profile="client"
servers=["10.10.A.A:9092", "10.10.B.B:9092", "10.10.C.C:9092", "10.10.D.D:9092", "10.10.E.E:9092", "10.10.F.F:9092"]
topic-refresh=60
offset-refresh=10

[consumer.production]
class-name="kafka"
cluster="prod-kafka"
servers=["10.10.A.A:9092", "10.10.B.B:9092", "10.10.C.C:9092", "10.10.D.D:9092", "10.10.E.E:9092", "10.10.F.F:9092"]
zookeeper-path="/local"
zookeeper-timeout=30
group-blacklist="^(console-consumer-).*$"
group-whitelist=""

[evaluator.mystorage]
class-name="caching"
expire-cache=60

[storage.default]
class-name="inmemory"
workers=20
intervals=10
expire-group=604800
min-distance=60

[httpserver.default]
address=":8080"`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the GET /v3/kafka/{cluster-name}/topic response after deleting topics, using the shown cluster configuration and topic-refresh setting. Trace Burrow’s topic discovery and refresh behavior to determine why deleted topics remain; done means the endpoint no longer reports them after refresh.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.