linkedin / linkedin/Burrow

consumer lag issue for slow consumers

Open
#658 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

I have a kafka producer which produces messages continuously but my consumer consumes every 2 seconds. Since my my consumer is slow consumer getting ERR notifications. find below configurations and API response.
**burrow.toml:**
[general]
pidfile="burrow.pid"
stdout-logfile="burrow.out"
access-control-allow-origin="mysite.example.com"

[logging]
filename="C://Go/src/Burrow/config/logs/burrow.log"
level="info"
maxsize=100
maxbackups=30
maxage=10
use-localtime=false
use-compression=true

[zookeeper]
servers=[ "localhost:2181" ]
timeout=6
root-path="/burrow"

[client-profile.test]
client-id="burrow-test"
kafka-version="1.1.1"

[cluster.df-test-app]
class-name="kafka"
servers=[ "localhost:9092" ]
client-profile="test"
topic-refresh=120
offset-refresh=30

[consumer.df-test-app]
class-name="kafka"
cluster="df-test-app"
servers=[ "localhost:9092" ]
client-profile="test"
group-denylist="^(console-consumer-|python-kafka-consumer-|quick-).*$"
group-allowlist=""

[cluster.df-test-clog]
class-name="kafka"
servers=[ "localhost:9092" ]
client-profile="test"
topic-refresh=120
offset-refresh=30

[consumer.df-test-clog]
class-name="kafka"
cluster="df-test-clog"
servers=[ "localhost:9092" ]
client-profile="test"
group-denylist="^(console-consumer-|python-kafka-consumer-|quick-).*$"
group-allowlist=""

[httpserver.default]
address=":8000"

[storage.default]
class-name="inmemory"
workers=20
intervals=5
expire-group=604800
min-distance=1

[notifier.default]
class-name="email"
interval=180
template-open="C://Go/src/Burrow/config/default-email.tmpl"
template-close="C://Go/src/Burrow/config/default-email-statusok.tmpl"
send-close=false
server="smtp.gmail.com"
port="587"
auth-type="plain"
username="test@gmail.com"
password="test123"
from="test@gmail.com"
to="test@gmail.com"
threshold=1

**burrow consumer group status:**
{
"error": false,
"message": "consumer status returned",
"status": {
"cluster": "df-test-app",
"group": "kvsrr.burrow.grp.3",
"status": "ERR",
"complete": 1,
"partitions": [
{
"topic": "kvsrr.burrow.topic",
"partition": 0,
"owner": "",
"client_id": "",
"status": "STOP",
"start": {
"offset": 14405033,
"timestamp": 1600009858736,
"observedAt": 1600046210000,
"lag": 147079641
},
"end": {
"offset": 14600596,
"timestamp": 1600009879054,
"observedAt": 1600046210000,
"lag": 146884078
},
"current_lag": 146884078,
"complete": 1
}
],
"partition_count": 1,
"maxlag": {
"topic": "kvsrr.burrow.topic",
"partition": 0,
"owner": "",
"client_id": "",
"status": "STOP",
"start": {
"offset": 14405033,
"timestamp": 1600009858736,
"observedAt": 1600046210000,
"lag": 147079641
},
"end": {
"offset": 14600596,
"timestamp": 1600009879054,
"observedAt": 1600046210000,
"lag": 146884078
},
"current_lag": 146884078,
"complete": 1
},
"totallag": 146884078
},
"request": {
"url": "/v3/kafka/df-test-app/consumer/kvsrr.burrow.grp.3/status",
"host": "IN01N03273"
}
}

Contributor guide

No contributing guide indexed for this repository

Research direction

The report provides burrow.toml and a Burrow consumer-status API response, but names no source file, test, or expected behavior. Start by reproducing the configuration and examining the /v3/kafka/df-test-app/consumer/.../status response for the ERR and STOP states. Done is not defined; the expected handling of a slow consumer must be established before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
observability-sre, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.