gojekfarm / gojekfarm/xtools

xkafka: Producer.AsyncPublish causes goroutine leak when Start/Run is not called

Open
#72 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
22
Forks
2
PR merge metrics
No merged PRs in 30d

Description

## Summary

The `Producer` struct requires `Start()` or `Run()` to be called to drain the internal `events` channel (delivery reports). However, `AsyncPublish` works without `Start()` being called — it silently pushes messages via `ProduceChannel()` without any compile-time or runtime indication that the event loop is missing.

## Problem

When `Start()` is not called:
1. The `events` channel ([producer.go#L17](https://github.com/gojekfarm/xtools/blob/main/xkafka/producer.go#L17)) fills up with unhandled delivery reports
2. The confluent-kafka-go internal poller blocks because it cannot push to the full events channel
3. `ProduceChannel()` backs up because the poller is blocked
4. Any goroutine calling `AsyncPublish` blocks forever on `p.kafka.ProduceChannel() <- km`
5. This results in an unbounded goroutine and memory leak

## Reproduction

Call `NewProducer()` followed by `AsyncPublish()` repeatedly without ever calling `Start()` or `Run()`. Observe goroutine count growing without bound.

## Suggested Improvements

- **Document the requirement**: The current doc on `Start()` only says "starts kafka event handling" without explaining it is mandatory for `AsyncPublish` to function correctly. Add explicit documentation that `Start()`/`Run()` must be called when using `AsyncPublish`.
- **Return an error**: Consider returning an error from `AsyncPublish` if `Start()` has not been called yet.
- **Auto-start**: Alternatively, auto-start the event loop lazily on the first `AsyncPublish` call via `sync.Once`.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read xkafka/producer.go around the events channel and the Start, Run, and AsyncPublish paths. Reproduce repeated AsyncPublish calls without Start or Run and observe the blocked producer behavior and goroutine growth. Done requires an agreed lifecycle behavior, clear documentation or API handling, and verification that AsyncPublish does not block indefinitely when the event loop is missing.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kafka
Domain
distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.