i/o timeout for __consumer_offsets topic
Open
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 4k
- Forks
- 818
- Avg merge
- 1h 14m
- Merged PRs (30d)
- 1
Description
Shortly after starting burrow starts throwing timeout error. I have built burrow just now with the latest code.
{"level":"error","ts":1588667263.4975047,"msg":"consume error","type":"module","coordinator":"consumer","class":"kafka","name":"mycluster","topic":"__consumer_offsets","partition":7,"error":"read tcp 172.17.0.7:44122->10.139.161.76:9093: i/o timeout"}
{"level":"error","ts":1588667263.4975176,"msg":"consume error","type":"module","coordinator":"consumer","class":"kafka","name":"mycluster","topic":"__consumer_offsets","partition":10,"error":"read tcp 172.17.0.7:44122->10.139.161.76:9093: i/o timeout"}
My burrow config:
[general]
pidfile="/workspace/burrow/burrow.pid"
stdout-logfile="/workspace/burrow/burrow.out"
access-control-allow-origin="*"
[logging]
filename="/workspace/burrow/logs/burrow.log"
level="info"
maxsize=50
maxbackups=5
#maxage=10
use-localtime=true
use-compression=true
[zookeeper]
servers=["broker1.xyz.com:2181", "broker3.xyz.com:2181", "broker2.xyz.com:2181"]
timeout=60
root-path="/burrow/notifier"
[client-profile.myprofile]
kafka-version="2.2.0"
sasl="mysaslprofile"
[sasl.mysaslprofile]
username="user"
password="mypwd"
handshake-first=true
[cluster.mycluster]
class-name="kafka"
servers=[ "broker1.xyz.com:9093", "broker3.xyz.com:9093", "broker2.xyz.com:9093" ]
client-profile="myprofile"
topic-refresh=120
offset-refresh=30
[consumer.mycluster]
class-name="kafka"
cluster="mycluster"
servers=[ "broker1.xyz.com:9093", "broker3.xyz.com:9093", "broker2.xyz.com:9093" ]
client-profile="myprofile"
start-latest=true
[httpserver.default]
address=":8000"
[storage.default]
class-name="inmemory"
workers=20
intervals=15
expire-group=604800
min-distance=1
[notifier.default]
....
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file, test, or entry point is named. Start by reproducing the logged i/o timeout using the supplied Burrow configuration, focusing on the Kafka consumer for the __consumer_offsets topic and the mycluster client profile. Done means identifying the cause and documenting or fixing the consumer timeout behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100