linkedin / linkedin/Burrow

Does burrow have a hard dependency on zookeeper?

Open
#534 9 comments 23 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

I'm wondering if it's possible to use burrow in a way that it doesn't depend on having access to zookeeper.

It looks like there is a [Consumer-Kafka](https://github.com/linkedin/Burrow/wiki/Consumer-Kafka) that can consume offsets directly from Kafka.

However, it looks like burrow may [depend on Zookeeper](https://github.com/linkedin/Burrow/wiki/Configuration#zookeeper) for other internal storage.

>The [zookeeper] heading specifies a Zookeeper ensemble to use in order to store metadata for modules and provide locking between multiple copies of Burrow. This allows you to run multiple copies of Burrow with only one being allowed to execute the notifiers at any time.

It also looks like the code will always attempt to setup a Zookeeper coordinator:

https://github.com/linkedin/Burrow/blob/a449cc412c48231a2b7347cd790f82b8ed18faf2/core/burrow.go#L36-L44

Is there a way to disable requiring zookeeper for a single-instance deployment of burrow?

Thanks for the great library!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with core/burrow.go at the linked coordinator setup and compare it with the Zookeeper configuration documented in the wiki. Check how Consumer-Kafka obtains offsets and determine whether a single-instance deployment can avoid the coordinator. Done means the supported behavior and any required configuration are clear and covered by appropriate documentation or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.