googleapis / googleapis/google-cloud-java

Cloud Storage SDK: Attempting to upload files larger than Java heap space results in OOM

Đang mở
#14,187 0 bình luận 0 reaction 1 người được giao Được @Dhriti07 nhận Xem trên GitHub
Ngôn ngữ chính
Java
Star
2.1k
Fork
1.2k
Merge trung bình
1 ngày 23 giờ
Pull request đã merge (30 ngày)
157

Mô tả

## Issue Details

Required: Provide a detailed description of the issue. Any additional information helps with figuring out the issue.

### Example
I am seeing {ISSUE} when running X, Y... after doing Z. This behavior started after the X library upgrade a few weeks ago ...

## Environment

Required: Provide details about your environment. If relevant, please also provide details about your GCP environment. The environment details MUST be filled out.
Provide the relevant details about your environment:

- OS: Fedora 44
- Java Version and JDK Vendor: Open JDK 17.0.17
- (If using GraalVM) GraalVM Version: N/A

(Optional) If deployed on GCP, provide the relevant details about your GCP deployment environment:

Storage test bench 0.54.0 running in podman

## Dependencies

Required: Provide a list of the dependencies that are being used. If unable to provide the list, please list the relevant libraries shown below. One of the two sections below MUST be filled out.

Provide the versions of the relevant libraries:
- Libraries-Bom ([com.google.cloud:libraries-bom](https://central.sonatype.com/artifact/com.google.cloud/libraries-bom)): N/A
- (If not using Libraries-Bom) Client Libraries:
- Gax (gax / gax-grpc / gax-httpjson): 2.83.0
- Auth (google-auth-library-oauth2-http / google-auth-library-credentials): 1.50.0
- Any other relevant Java SDK dependencies:
- Google-Http-Java-Client ([com.google.http-client:google-http-client](https://central.sonatype.com/artifact/com.google.http-client/google-http-client)): 2.2.0

## Reproducer

Required: Provide a reproducer and the steps needed to reproduce this issue locally. If unable to provide a reproducer, please provide code snippets to help reproduce the issue. One of the two sections below MUST be filled out.

Provide as many code snippets as possible:

### Example
Create transfer manager:
```java
public GoogleStorageClient(StorageClientConfig clientConfig) {
config = new GoogleStorageClientConfig(clientConfig);
GoogleCredentials credentials = config.googleCredentials;
if(credentials instanceof AnonymousGoogleCredentials) {
storage = StorageOptions.newBuilder().setCredentials(credentials).setHost(clientConfig.getEndpointUrl())
.setTransportOptions(HttpTransportOptions.newBuilder().setConnectTimeout(GoogleCloudConfig.GOOGLE_REQUEST_TIMEOUT).build())
.setProjectId(config.projectId).build().getService();
} else {
storage = StorageOptions.newBuilder().setCredentials(credentials).setProjectId(config.projectId).setTransportOptions(HttpTransportOptions.newBuilder().setConnectTimeout(GoogleCloudConfig.GOOGLE_REQUEST_TIMEOUT).build()).build().getService();
}
transferManager = TransferManagerConfig.newBuilder().setAllowDivideAndConquerDownload(true).setAllowParallelCompositeUpload(true).setStorageOptions(storage.getOptions()).build().getService();
}
```
Same class
```java
public uploadFiles(File[] sourceFiles, ) {
ParallelUploadConfig parallelUploadConfig = ParallelUploadConfig.newBuilder().setBucketName(bucketName).build();
List results =
transferManager.uploadFiles(files, parallelUploadConfig).getUploadResults();

}
```

Create a client and attempt to upload a large file, larger than the JVM internal memory. I tested with a 3.2 GB file with 512 MB heap. The upload fails at the `uploadFiles` call with
```
Java heap spacecom.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1383)
at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1376)
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53)
at com.google.cloud.storage.transfermanager.UploadJob.getUploadResults(UploadJob.java:57)
at com.ibm.cmus.google_cloud_storage.utils.client.GoogleStorageClient.uploadFiles(GoogleStorageClient.java:436)
```

I develop a tool which provides methods for interacting with various storage vendors (i.e. s3). I don't see other transfer managers struggling with memory and I am able to upload the same file without issues. While the simple fix is to increase heap space, as that size is obviously quite small, it is enough for other vendors to upload the same file.

## Logs and Stack Trace

Required: Provide logs that showcase the error. This will help show the flow of the application and help narrow down the cause. Additionally, provide a stack trace of the error if possible.

FYI, the below link is broken, the troubleshooting guide appears to have been removed sometime in the last 2 years.
The Java SDK has a troubleshooting [guide](https://github.com/googleapis/google-cloud-java/blob/main/TROUBLESHOOTING.md) for enabling logs. This contains information regarding client-server communication, request and response details, and logging in dependency libraries. If using this guide, please obfuscate any private information (bearer tokens, request and response params, etc).

Additionally, please provide a stack trace of the error seen:

### Exception
```
Java heap spacecom.google.common.util.concurrent.ExecutionError: java.lang.OutOfMemoryError: Java heap space
at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1383)
at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1376)
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:53)
at com.google.cloud.storage.transfermanager.UploadJob.getUploadResults(UploadJob.java:57)
at com.ibm.cmus.google_cloud_storage.utils.client.GoogleStorageClient.uploadFiles(GoogleStorageClient.java:436)
```

## Behavior

Optional: Any additional information about the behavior of the error is helpful to debug.

Behavioral Questions:

- When did the issue begin? Is this behavior related to any dependency version upgrade? - I have not tested files of this size on older versions
- Is this behavior flaky? Or is this consistently seen in production? - This is consistently seen. The memory requirements for uploading files grow with file size.
- Is this behavior related to the volume of data? - No, but it is related to the size of the file.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.