killme2008 / killme2008/xmemcached
XMemcached pooled client Recv Q size is very high
- Dominant language
- Java
- Stars
- 764
- Forks
- 271
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
We've been using XMemcachedClient connected to twemproxy which sits behind set of memcached servers. We've kept a OpTimeout of 30 ms.
Over the time, the Recv Q(from netstat) size of the tcp socket grows really high and prompting timeouts from XMemcached. Twemproxy as a result is also getting killed by going OOM gradually
Is XMemcached client slow in receiving data? Should `TCP_RECV_BUFF_SIZE` be increased?
Few more details:
1. XMemcached version: 2.4.0
2. Protocol: Text Protocol
3. Session Idle Timeout: 60 secs
4. timeoutExceptionThreshold: 250000
5. healSessionInterval: 1 sec
6. Pool Size: 20
7. Throughput: ~ 2.2K QPS
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the XMemcachedClient pooled-client behavior described for version 2.4.0, using the twemproxy, text-protocol, 30 ms timeout, and 2.2K QPS conditions as the reproduction context. Inspect the TCP receive queue, timeout behavior, and TCP_RECV_BUFF_SIZE handling; done means identifying the cause of queue growth and documenting or implementing a validated fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, memcached
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100