linkedin / linkedin/Burrow

Is Zookeeper mandated for using Burrow?

Open
#811 2 comments 5 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

Hi Team,

I am trying to use Burrow for my Kafka cluster but I didn't provide Zookeeper configuration since I might switch to KRaft mode soon and thought it was not mandatory to use Zookeeper.

My setup failed and my pod on K8s couldn't start and entered a CrashLoopBackOff issue and found the following in the pod logs:

```
Reading configuration from /etc/burrow
panic: No Zookeeper servers specified [recovered]
panic: No Zookeeper servers specified [recovered]
panic: No Zookeeper servers specified

goroutine 1 [running]:
main.handleExit()
/home/runner/work/Burrow/Burrow/main.go:63 +0xf8
panic({0xbeb8a0, 0xc000210a30})
/opt/hostedtoolcache/go/1.20.1/x64/src/runtime/panic.go:884 +0x213
go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x0?, 0xc0001001a0?, {0x0?, 0x0?, 0xc00021a700?})
/home/runner/go/pkg/mod/go.uber.org/zap@v1.24.0/zapcore/entry.go:198 +0x65
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc000200b60, {0x0, 0x0, 0x0})
/home/runner/go/pkg/mod/go.uber.org/zap@v1.24.0/zapcore/entry.go:264 +0x3ec
go.uber.org/zap.(*Logger).Panic(0x0?, {0xd26f47?, 0x0?}, {0x0, 0x0, 0x0})
/home/runner/go/pkg/mod/go.uber.org/zap@v1.24.0/logger.go:258 +0x59
github.com/linkedin/Burrow/core.configureCoordinators.func1()
/home/runner/work/Burrow/Burrow/core/burrow.go:96 +0x56
panic({0xbeb8a0, 0xebb050})
/opt/hostedtoolcache/go/1.20.1/x64/src/runtime/panic.go:884 +0x213
github.com/linkedin/Burrow/core/internal/zookeeper.(*Coordinator).Configure(0xc000232400)
/home/runner/work/Burrow/Burrow/core/internal/zookeeper/coordinator.go:64 +0x26b
github.com/linkedin/Burrow/core.configureCoordinators(0xc0002203c0, {{0xec26e0, 0xc000232400}, {0xec26b0, 0xc000206c60}, {0xec2500, 0xc00021a660}, {0xec25c0, 0xc000206cf0}, {0xec25f0, ...}, ...})
/home/runner/work/Burrow/Burrow/core/burrow.go:103 +0xb3
github.com/linkedin/Burrow/core.Start(0xc000220360?, 0xc00012bef0?)
/home/runner/work/Burrow/Burrow/core/burrow.go:151 +0x3dc
main.main()
/home/runner/work/Burrow/Burrow/main.go:114 +0x4d2
```

Does this mean I can't use Burrow without providing a Zookeeper config? What if I am using KRaft?

> **Burrow image used:** ghcr.io/linkedin/burrow/burrow:v1.6.0

> **Configurations**

```
[logging]
filename="logs/burrow.log"

[client-profile.myclient]
kafka-version="3.6.1"
client-id="borrow-client"
sasl="saslprofile"

[sasl.saslprofile]
username="myuser"
password="mypassword"
mechanism="SCRAM-SHA-512"

[httpserver.mylistener]
address=":8080"

[cluster.myclustername]
class-name="kafka"
servers=[ "my.single.server" ]
client-profile="myclient"
topic-refresh=60
offset-refresh=30
groups-reaper-refresh=30

[consumer.myconsumers]
class-name="kafka"
cluster="myclustername"
servers=[ "my.single.server" ]
client-profile="myclient"
offsets-topic="__consumer_offsets"
start-latest=true
group-allowlist=".*"
```

I will appreciate any help from the team or the community, TIA :")

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with core/internal/zookeeper/coordinator.go and the coordinator setup in core/burrow.go, using the provided panic trace as the entry point. Determine whether Burrow's current configuration supports a Kafka cluster without ZooKeeper, including KRaft, and define the compatibility or implementation work needed before this can be considered done.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.