InProcess: client streaming memory leak when retry enabled
- Dominant language
- Java
- Stars
- 12.1k
- Forks
- 4k
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 37
Description
### What version of gRPC-Java are you using?
1.42.0
### What is your environment?
Linux 5.15.2
openjdk version "11.0.13" 2021-10-19
### What did you expect to see?
No java heap space exception.
### What did you see instead?
`java.lang.OutOfMemoryError: Java heap space`
### Steps to reproduce the bug
We've recently updated our grpc dependency from 1.39.0 to 1.42.0 which now causes an out of memory error in one of our tests.
I've attached a test project that can reproduce the problem.
[grpcheap.zip](https://github.com/grpc/grpc-java/files/7562661/grpcheap.zip)
How to run:
```
mvn clean test -Dgrpc.version=1.39.0 # test runs successfully
mvn clean test -Dgrpc.version=1.42.0 # test fails with OOM
```
Please be aware that the test is run with limited heap size (see pom.xml): `-Xmx64M`
The general idea of the test is this:
Client-side has an `InputStream` that produces ~200mb data. This data of that stream is (lazily) split into ~1mb chunks and these are then streamed to the server.
What happens when I run the test with grpc >1.40 is that I get an OOM when it has sent ~60mb worth of chunks, which obviously is near the `-Xmx` limit. However, what I would expect is that the heap usage would never rise to that level, as after each 1mb chunk has been sent that chunk can be gc'ed. I've also attached a screenshot of visualvm where you can see the heap graph of the test using grpc 1.39.0 vs 1.42.0:

Contributor guide
Assessment
This issue has not been assessed yet.