Burrow does not seem to work with offsets stored in Zookeeper
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 4k
- Forks
- 818
- Avg merge
- 1h 14m
- Merged PRs (30d)
- 1
Description
Hi everybody,
I have these Kafka brokers writing offsets to a Zookeeper cluster.
Here is burrow.toml:
```
[SRV_MONRTDM@pmonigovas01v ~]$ cat burrow30/burrow.toml
[general]
access-control-allow-origin="*" #Header value to put in http response
[zookeeper]
servers=["10.207.180.133:2181", "10.207.180.134:2181", "10.207.180.135:2181" ]
timeout=6 #expiration timeout for zookeeper sessions.
root-path="/burrow" #full path to zookeeper node that burrow will be allowed to write into.
[client-profile.myclient] #this client profile name is myclient
kafka-version="0.10" #kafka server version
client-id="burrow-client" # a string to be passed to kafka as client Id
[httpserver.mylistener]
address=":8000" #port to send http request
[storage.mystorage]
class-name="inmemory"
intervals=10 #no. of offsets to store for each partition
expire-group=604800 # no. of seconds after which a group will be purged if it has not commited offset
[cluster.hdp30]
class-name="kafka"
servers=[ "10.207.180.142:6667", "10.207.180.143:6667", "10.207.180.144:6667", "10.207.180.145:6667" ]
client-profile="myclient" #profile name defined as in client profile subheading.
topic-refresh=10
offset-refresh=10
[consumer.hdp-burrow30]
class-name="kafka_zk"
servers=["10.207.180.133:2181", "10.207.180.134:2181", "10.207.180.135:2181" ]
cluster="hdp30" # sub heading name defined in above cluster configuration
zookeeper-timeout=5
zookeeper-path=""
```
No consumers show in burrow-ui, burrow-dashboard or by curling burrow HTTP endpoint:
```
$ docker logs -f burrow
Reading configuration from /etc/burrow
{"level":"info","ts":1561039131.3334801,"msg":"Started Burrow"}
{"level":"info","ts":1561039131.333604,"msg":"configuring","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1561039131.333678,"msg":"configuring","type":"coordinator","name":"storage"}
{"level":"info","ts":1561039131.3337097,"msg":"configuring","type":"module","coordinator":"storage","class":"inmemory","name":"mystorage"}
{"level":"info","ts":1561039131.3338077,"msg":"configuring","type":"coordinator","name":"evaluator"}
{"level":"info","ts":1561039131.3338366,"msg":"configuring","type":"module","coordinator":"evaluator","class":"caching","name":"default"}
{"level":"info","ts":1561039131.3338754,"msg":"configuring","type":"coordinator","name":"httpserver"}
{"level":"info","ts":1561039131.333965,"msg":"configuring","type":"coordinator","name":"notifier"}
{"level":"info","ts":1561039131.334004,"msg":"configuring","type":"coordinator","name":"cluster"}
{"level":"info","ts":1561039131.33403,"msg":"configuring","type":"module","coordinator":"cluster","class":"kafka","name":"hdp30"}
{"level":"info","ts":1561039131.3341236,"msg":"configuring","type":"coordinator","name":"consumer"}
{"level":"info","ts":1561039131.3341634,"msg":"configuring","type":"module","coordinator":"consumer","class":"kafka_zk","name":"hdp-burrow30"}
{"level":"info","ts":1561039131.3342574,"msg":"starting","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1561039131.3352957,"msg":"Connected to 10.207.180.134:2181","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1561039131.336034,"msg":"Authenticated: id=174272131742600143, timeout=6000","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1561039131.3360665,"msg":"Re-submitting `0` credentials after reconnect","type":"coordinator","name":"zookeeper"}
{"level":"info","ts":1561039131.3410175,"msg":"starting","type":"coordinator","name":"storage"}
{"level":"info","ts":1561039131.3410745,"msg":"starting","type":"module","coordinator":"storage","class":"inmemory","name":"mystorage"}
{"level":"info","ts":1561039131.3413737,"msg":"starting","type":"coordinator","name":"evaluator"}
{"level":"info","ts":1561039131.341399,"msg":"starting","type":"module","coordinator":"evaluator","class":"caching","name":"default"}
{"level":"info","ts":1561039131.3414104,"msg":"starting","type":"coordinator","name":"httpserver"}
{"level":"info","ts":1561039131.3640947,"msg":"started listener","type":"coordinator","name":"httpserver","listener":"0.0.0.0:8000"}
{"level":"info","ts":1561039131.364167,"msg":"starting","type":"coordinator","name":"notifier"}
{"level":"info","ts":1561039131.3641877,"msg":"starting","type":"coordinator","name":"cluster"}
{"level":"info","ts":1561039131.3641932,"msg":"starting","type":"module","coordinator":"cluster","class":"kafka","name":"hdp30"}
{"level":"info","ts":1561039131.402523,"msg":"starting","type":"coordinator","name":"consumer"}
{"level":"info","ts":1561039131.4025538,"msg":"starting","type":"module","coordinator":"consumer","class":"kafka_zk","name":"hdp-burrow30"}
{"level":"info","ts":1561039131.4047937,"msg":"Connected to 10.207.180.134:2181","type":"module","coordinator":"consumer","class":"kafka_zk","name":"hdp-burrow30"}
{"level":"info","ts":1561039131.4054527,"msg":"Authenticated: id=174272131742600144, timeout=6000","type":"module","coordinator":"consumer","class":"kafka_zk","name":"hdp-burrow30"}
{"level":"info","ts":1561039131.4054873,"msg":"Re-submitting `0` credentials after reconnect","type":"module","coordinator":"consumer","class":"kafka_zk","name":"hdp-burrow30"}
{"level":"info","ts":1561039131.4660237,"msg":"starting evaluations","type":"coordinator","name":"notifier"}
{"level":"info","ts":1561039131.6079664,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-59921","showall":true}
{"level":"info","ts":1561039131.6086192,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-66028","showall":true}
{"level":"info","ts":1561039131.6089625,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-17809","showall":true}
{"level":"info","ts":1561039131.6092784,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-17714","showall":true}
{"level":"info","ts":1561039131.6096077,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-91917","showall":true}
{"level":"info","ts":1561039131.610201,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-89795","showall":true}
{"level":"info","ts":1561039131.6121168,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-68185","showall":true}
{"level":"info","ts":1561039131.6126003,"msg":"cluster or consumer not found","type":"module","coordinator":"evaluator","class":"caching","name":"default","cluster":"hdp30","consumer":"console-consumer-94287","showall":true}
```
So, question is: is it because offsets are not being written in __consumer_offset ?
Thanks
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the kafka_zk consumer module and its handling of the configured Zookeeper paths, then compare the reported configuration with where Kafka stores consumer offsets. Reproduce the setup using the supplied burrow.toml and logs; done means the configured consumers appear through the Burrow HTTP endpoint and UI.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100