eclipse-ee4j / eclipse-ee4j/tyrus
Gradually Increasing Memory Usage with Tyrus Standalone Client 1.11
- Dominant language
- Java
- Stars
- 128
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
Setup: I have a long running Java 8 service that establishes several persistent websocket connections to a Tomcat 7 websocket server. Messages are sent and received at a fairly slow, consistent rate (3-15 every 15 seconds). The websocket connections disconnect and reconnect about every 10 minutes. I have set the max heap size to 40Mb, just to more efficiently troubleshoot OutOfMemory errors.
Problem: Eventually (after about 2 days), the java process begins throwing `java.lang.OutOfMemoryError: GC overhead limit exceeded`.
Monitoring the rss value of the process via the ps command shows the memory usage of the process increasing gradually (~2k/minute).
Monitoring the memory usage with jstat shows old generation usage increase gradually until it hits the limit. When it does, a full garbage collection happens and cleans up most of it, but each time it leaves a small amount. Eventually it reaches the GC limit threshold and throws the above error.
Analyzing the heap with jmap/jhat, and viewing instance counts shows 4 at the very top (in this order): org.glassfish.grizzly.http.util.BufferChunk, org.glassfish.grizzly.http.util.ByteChunk, org.glassfish.grizzly.http.util.CharChunk, org.glassfish.grizzly.http.util.DataChunk. These instances only get cleaned up when a full garbage collection happens, but each time it leaves some of them.
Viewing the heap histogram shows instances of [B consuming very large amounts (~50%) of the max heap size.
This seems similar to [GRIZZLY-84](https://java.net/jira/browse/GRIZZLY-84 "Need leak-free bytebuffer management in Grizzly"), but that is marked as already fixed.
#### Environment
Tyrus Standalone Client 1.11
Java 8 (-Xmx40M)
Debian Jessie
Tomcat 7
#### Affected Versions
[1.11]
Contributor guide
Assessment
This issue has not been assessed yet.