kakao / kakao/varlog

client: fix panic caused by too large channel size in pkg/varlog.subscribedLogEntriesQueue

Open
#653 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
53
Forks
7
PR merge metrics
No merged PRs in 30d

Description

Subscribe API creates a queue to keep log entries received from each log stream. The queue, subscribedLogEntriesQueue, uses a buffered channel as an MPSC queue. Since its buffer size is as large as the subscribe span length, it can panic if the span is too broad. For example, users can subscribe to log entries without stopping by passing MaxGLSN as an end position of range to the Subscribe API. However, it makes the channel size too large, so panic occurs.

https://github.com/kakao/varlog/blob/9e6713b54783c60b0ac13f3ba858d81ed11039a2/pkg/varlog/subscribe.go#L452-L461

It is essential to ensure that the queue length is not too large, regardless of the subscription range. The queue size should be small enough to avoid wasting memory but not so small that it requires frequent calls to the Subscribe API. We may need to refactor the overall architecture to address this issue.

Contributor guide

Open the contributing guide

Research direction

Start with pkg/varlog/subscribe.go at the linked lines and trace subscribedLogEntriesQueue as used by the Subscribe API. Check how a MaxGLSN end position turns the subscription span into a channel size. Done means broad or unbounded subscriptions no longer panic or waste excessive memory while retaining practical queue behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.