client: fix panic caused by too large channel size in pkg/varlog.subscribedLogEntriesQueue
- Dominant language
- Go
- Stars
- 53
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
Subscribe API creates a queue to keep log entries received from each log stream. The queue, subscribedLogEntriesQueue, uses a buffered channel as an MPSC queue. Since its buffer size is as large as the subscribe span length, it can panic if the span is too broad. For example, users can subscribe to log entries without stopping by passing MaxGLSN as an end position of range to the Subscribe API. However, it makes the channel size too large, so panic occurs.
https://github.com/kakao/varlog/blob/9e6713b54783c60b0ac13f3ba858d81ed11039a2/pkg/varlog/subscribe.go#L452-L461
It is essential to ensure that the queue length is not too large, regardless of the subscription range. The queue size should be small enough to avoid wasting memory but not so small that it requires frequent calls to the Subscribe API. We may need to refactor the overall architecture to address this issue.
Contributor guide
Research direction
Start with pkg/varlog/subscribe.go at the linked lines and trace subscribedLogEntriesQueue as used by the Subscribe API. Check how a MaxGLSN end position turns the subscription span into a channel size. Done means broad or unbounded subscriptions no longer panic or waste excessive memory while retaining practical queue behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100