linkedin / linkedin/Burrow

Connector lag monitoring support

Open
#515 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

Hi,

I'm currently giving a try with Burrow and running into some problems.
I'm trying to monitor lag for connectors (HDFS, JDBC, etc) and when I'm starting burrow on my test environment I get the following debug trace :

```
{"level":"debug","ts":1554735418.3128974,"msg":"skipped metadata because of unknown protocolType","type":"module","coordinator":"consumer","class":"kafka","name":"consumer_kafka","offset_topic":"__consumer_offsets","offset_partition":3,"offset_offset":1117,"message_type":"metadata","group":"connector.dev._connect_topic-namexxxx-mz0zx7t8u7-cluster","protocol_type":"connect","generation":1,"protocol":"default","leader":"connect-1-d4fcb6c5-91af-4f6a-a735-ad82fd748c36","current_state_timestamp":1553520383214}
{"level":"debug","ts":1554735418.3129158,"msg":"skipped metadata because of unknown protocolType","type":"module","coordinator":"consumer","class":"kafka","name":"consumer_kafka","offset_topic":"__consumer_offsets","offset_partition":3,"offset_offset":1118,"message_type":"metadata","group":"connector.dev._connect_topic-namexx-sourcexxxx-mz0zx7t8u7-cluster","protocol_type":"connect","generation":2,"protocol":"default","leader":"connect-1-d4fcb6c5-91af-4f6a-a735-ad82fd748c36","current_state_timestamp":1553520384339}
```
So I guess `connect` is not a supported protocolType ?
I see that this code was added 3 months ago and apparently is the cause of this `pruning`.
https://github.com/linkedin/Burrow/blob/3cdc7e73b8388985d95ea7f05ef23c7bec3e9985/core/internal/consumer/kafka_client.go#L433-L437

Is there any plan to add this support in the future ? or this is more a PEBKAC ?

FYI : I also tested to remove this code and I got another error when extracting the group metadata. It fail with a `"failed to decode"` `reason:assigment` here :
https://github.com/linkedin/Burrow/blob/3cdc7e73b8388985d95ea7f05ef23c7bec3e9985/core/internal/consumer/kafka_client.go#L588-L591

And caused by an error in the member assignment extraction : https://github.com/linkedin/Burrow/blob/3cdc7e73b8388985d95ea7f05ef23c7bec3e9985/core/internal/consumer/kafka_client.go#L611-L614

But I'm not sure that those two errors are related.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in core/internal/consumer/kafka_client.go, especially the protocolType handling around lines 433-437 and group metadata decoding around lines 588-614. Reproduce the connector-group trace, then determine whether the connect protocol can be supported without the reported metadata and member-assignment decoding failures. Done means connector lag groups are handled without being pruned or failing to decode.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.