linkedin / linkedin/Burrow

Delete groups with no topics

Open
#426 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
4k
Forks
818
Avg merge
1h 14m
Merged PRs (30d)
1

Description

The current policy (AFAIK) is that consumer groups do not get deleted unless they expire or are manually deleted via the REST API.

Kafka topics are periodically removed from the storage coordinator when the sarama client metadata is refreshed. However, after deletion of the topic there is no test to see if any their parent consumer groups still have one or more child topics.

This leads to a situation where a consumer group is present in the list of consumers but when its status is fetched from the evaluator a StatusOK response is returned but with no partitions.

I would propose (at least for now) testing for empty consumer groups and deleting them when deleting topics.

I have a test for this in my PR for fixing the div. by zero bug: https://github.com/linkedin/Burrow/pull/411

Relates to: https://github.com/linkedin/Burrow/issues/265 https://github.com/linkedin/Burrow/issues/410

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing how topics are removed from the storage coordinator and the test for this behavior referenced in PR #411. Trace how consumer groups retain child topics after deletion, then verify that groups with no topics are removed and no longer appear with an empty status.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.