UploadPartResponse seems to hold a reference to the content of the part, even after stream is closed
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 51
Description
### Describe the bug
When using the `S3Client` to `uploadPart`, the returned `UploadPartResponse` seems to hold a reference to the content of the part, causing memory build up over time
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Expected Behavior
The `UploadPartResponse` should only hold information related to the actual response as described here: https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html#API_UploadPart_ResponseElements
### Current Behavior
When uploading multiple parts in sequence and keeping the `UploadPartResponse` in memory to be used later to complete the multi part upload we see a build up of memory over time. Upon inspection of a heap dump we noticed that the `UploadPartResponse` had a memory footprint roughly equal to our part size, which was unexpected.
It seems that `UploadPartResponse` holds a reference to an underlying `sdkHttpResponse` which in turn holds a `content` reference which is an `AbortableInputStream` the size of the part.
### Reproduction Steps
Our code is written in Scala, but to the best of my knowledge this Java code serves as a minimum example
```java
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.*;
import software.amazon.awssdk.core.sync.RequestBody;
import java.io.ByteArrayInputStream;
import java.util.ArrayList;
import java.util.List;
public class S3MultipartUploadFullExample {
public static void main(String[] args) {
S3Client s3 = S3Client.create();
String bucket = "your-bucket";
String key = "your-key";
byte[] chunk = new byte[50 * 1024 * 1024]; // 50MB chunk
CreateMultipartUploadRequest createRequest = CreateMultipartUploadRequest.builder()
.bucket(bucket)
.key(key)
.build();
CreateMultipartUploadResponse initiateResult = s3.createMultipartUpload(createRequest);
// We need to hold the responses so we can complete the multi part upload later
List partUploadResponses = new ArrayList<>();
for (int i = 0; i < 100; i++) {
try (ByteArrayInputStream is = new ByteArrayInputStream(chunk)) {
int partNumber = i + 1;
UploadPartRequest uploadPartRequest = UploadPartRequest.builder()
.bucket(bucket)
.key(key)
.uploadId(initiateResult.uploadId())
.partNumber(partNumber)
.contentLength(chunk.length)
.build();
UploadPartResponse response = s3.uploadPart(uploadPartRequest, RequestBody.fromInputStream(is, chunk.length));
partUploadResponses.add(response);
}
}
// At this point the heap has grown to 100 * 50MB ~ 5GB because each UploadPartResponse
// in partUploadResponses holds a reference to a stream with the content
List completedParts = new ArrayList<>();
for (int i = 0; i < partUploadResponses.size(); i++) {
completedParts.add(CompletedPart.builder()
.partNumber(i + 1)
.eTag(partUploadResponses.get(i).eTag())
.build());
}
CompletedMultipartUpload completedMultipartUpload = CompletedMultipartUpload.builder()
.parts(completedParts)
.build();
CompleteMultipartUploadRequest completeRequest = CompleteMultipartUploadRequest.builder()
.bucket(bucket)
.key(key)
.uploadId(initiateResult.uploadId())
.multipartUpload(completedMultipartUpload)
.build();
CompleteMultipartUploadResponse completeResponse = s3.completeMultipartUpload(completeRequest);
System.out.println("Multipart upload completed: " + completeResponse.key());
s3.close();
}
}
```
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### AWS Java SDK version used
2.31.78
### JDK version used
Picked up JAVA_TOOL_OPTIONS: -XX:UseSVE=0 openjdk 21.0.7 2025-04-15 OpenJDK Runtime Environment (build 21.0.7+6-alpine-r0) OpenJDK 64-Bit Server VM (build 21.0.7+6-alpine-r0, mixed mode, sharing)
### Operating System and version
Linux Alpine 3.21.3
Contributor guide
Assessment
This issue has not been assessed yet.