Delete groups with no topics
- Dominant language
- Go
- Stars
- 4k
- Forks
- 818
- Avg merge
- 1h 14m
- Merged PRs (30d)
- 1
Description
The current policy (AFAIK) is that consumer groups do not get deleted unless they expire or are manually deleted via the REST API.
Kafka topics are periodically removed from the storage coordinator when the sarama client metadata is refreshed. However, after deletion of the topic there is no test to see if any their parent consumer groups still have one or more child topics.
This leads to a situation where a consumer group is present in the list of consumers but when its status is fetched from the evaluator a StatusOK response is returned but with no partitions.
I would propose (at least for now) testing for empty consumer groups and deleting them when deleting topics.
I have a test for this in my PR for fixing the div. by zero bug: https://github.com/linkedin/Burrow/pull/411
Relates to: https://github.com/linkedin/Burrow/issues/265 https://github.com/linkedin/Burrow/issues/410
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing how topics are removed from the storage coordinator and the test for this behavior referenced in PR #411. Trace how consumer groups retain child topics after deletion, then verify that groups with no topics are removed and no longer appear with an empty status.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100