OutOfMemoryError with S3TransferManager and S3CrtAsyncClient

### Describe the bug

We have an OpenSource Java application that uses the S3TransferManager and S3CrtAsyncClient, the [OpenSearch Migration Assistant](https://github.com/opensearch-project/opensearch-migrations/).

We have observed some OutOfMemoryErrors when downloading s3 directories with the transfer manager.

While I do not have consistent reproduction steps, I followed advice from @alextwoods regarding limiting transfer manager executor to no avail.

Scenario:
Downloading directory with `S3TransferManager#downloadDirectory`
Directory consists of 132 files with 53 files between 125MB and 264MB, 12 files from 3MB to 80MB. 2 files ~40KB and 65 files that are 479B

Configuration:
Client:
```
S3AsyncClient s3Client = S3AsyncClient.crtBuilder()
.region(Region.of(s3Region))
.credentialsProvider(DefaultCredentialsProvider.builder().build())
.retryConfiguration(r -> r.numRetries(3))
.targetThroughputInGbps(8.0)
.maxNativeMemoryLimitInBytes(1073741824L) // 1 GiB
.minimumPartSizeInBytes(8388608L) // 8MiB
.maxConcurrency(RANDOM_SIZE) // (1 - 500, explained later)
.endpointOverride(s3Endpoint)
```

Transfer Manager:
```
S3TransferManager.builder()
.s3Client(s3Client)
.executor(executor)
.build())
```
Executor:
```
// Modified from TransferManagerConfiguration#defaultExecutor
ThreadPoolExecutor executor = new ThreadPoolExecutor(0, RANDOM_POOL_SIZE, // Pool size between 1 - 500
60, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(1_000),
new ThreadFactoryBuilder()
.threadNamePrefix("rfs-s3-transfer-manager").build());
// Allow idle core threads to time out
executor.allowCoreThreadTimeOut(true);
```

I ran 41 downloads over the course of 12 hours, total download is 9.2GB on each attempt. Out of the 42 downloads, 7 failed with OOM (logs gathered included below).

I ran with random pool size and maxConcurrency out of options: `(1, 5, 10, 20, 50, 100, 200, 500)` (independently chosen) and observed the 7 failures with the following configurations:

* max concurrency 5 pool 50
* max concurrency 50 pool 100
* max concurrency 5 pool 5
* max concurrency 5 pool 500
* max concurrency 5 pool 500
* max concurrency 10 pool 10

Environment:
Base Image: `amazoncorretto:17-al2023-headless`
Architecture: ARM64
Machine: Fargate 2vCPU 4gb Memory
Disk: EBS GP3 (~500GB)
Java Options: `-XX:MaxRAMPercentage=65.0` (2.6 GB)
AWS Region: US-EAST-1
Library Version: `aws-crt = "0.38.7"`, `aws-sdk = "2.32.4"`

Exception:
```

2025-08-08 06:52:50,922 INFO o.o.m.b.c.S3Repo [main] Downloading blob files from S3 with max concurrency 5 pool 5: s3://migration-artifacts-REDACTED-us-east-1/newsnap/indices/PYO6TRrfSyuReHLFwBDXQw/3/ to /storage/s3_files/indices/PYO6TRrfSyuReHLFwBDXQw/3
--
Exception in thread "AwsEventLoop 2" java.lang.OutOfMemoryError: Java heap space
2025-08-08 06:53:22,198 WARN s.a.a.s.s.i.c.S3CrtResponseHandlerAdapter [Thread-34] Exception thrown in responsePublisher#error, ignoringjava.util.concurrent.CancellationException: subscription has been cancelled. at software.amazon.awssdk.utils.async.SimplePublisher.lambda$doProcessQueue$9(SimplePublisher.java:286) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.get(SimplePublisher.java:413) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.access$700(SimplePublisher.java:397) at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:260) at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224) at software.amazon.awssdk.utils.async.SimplePublisher.access$1300(SimplePublisher.java:58) at software.amazon.awssdk.utils.async.SimplePublisher$SubscriptionImpl.cancel(SimplePublisher.java:389) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:244) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:223) at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:131) at java.base/sun.nio.ch.Invoker$3.run(Invoker.java:241) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)Exception in thread "AwsEventLoop 1" java.lang.OutOfMemoryError: Java heap spaceException in thread "AwsEventLoop 2" java.lang.OutOfMemoryError: Java heap spaceException in thread "AwsEventLoop 2" java.lang.OutOfMemoryError: Java heap space
2025-08-08 06:53:25,110 WARN s.a.a.s.s.i.c.S3CrtResponseHandlerAdapter [Thread-32] Exception thrown in responsePublisher#error, ignoringjava.util.concurrent.CancellationException: subscription has been cancelled. at software.amazon.awssdk.utils.async.SimplePublisher.lambda$doProcessQueue$9(SimplePublisher.java:286) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.get(SimplePublisher.java:413) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.access$700(SimplePublisher.java:397) at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:260) at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224) at software.amazon.awssdk.utils.async.SimplePublisher.access$1300(SimplePublisher.java:58) at software.amazon.awssdk.utils.async.SimplePublisher$SubscriptionImpl.cancel(SimplePublisher.java:389) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:244) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:223) at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:131) at java.base/sun.nio.ch.Invoker$3.run(Invoker.java:241) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)
2025-08-08 06:53:26,500 WARN s.a.a.s.s.i.c.S3CrtResponseHandlerAdapter [Thread-14] Exception thrown in responsePublisher#error, ignoringjava.util.concurrent.CancellationException: subscription has been cancelled. at software.amazon.awssdk.utils.async.SimplePublisher.lambda$doProcessQueue$9(SimplePublisher.java:286) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.get(SimplePublisher.java:413) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.access$700(SimplePublisher.java:397) at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:260) at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224) at software.amazon.awssdk.utils.async.SimplePublisher.access$1300(SimplePublisher.java:58) at software.amazon.awssdk.utils.async.SimplePublisher$SubscriptionImpl.cancel(SimplePublisher.java:389) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:244) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:223) at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:131) at java.base/sun.nio.ch.Invoker$3.run(Invoker.java:241) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)
2025-08-08 06:53:28,271 WARN s.a.a.s.s.i.c.S3CrtResponseHandlerAdapter [Thread-10] Exception thrown in responsePublisher#error, ignoringjava.util.concurrent.CancellationException: subscription has been cancelled. at software.amazon.awssdk.utils.async.SimplePublisher.lambda$doProcessQueue$9(SimplePublisher.java:286) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.get(SimplePublisher.java:413) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.access$700(SimplePublisher.java:397) at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:260) at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224) at software.amazon.awssdk.utils.async.SimplePublisher.access$1300(SimplePublisher.java:58) at software.amazon.awssdk.utils.async.SimplePublisher$SubscriptionImpl.cancel(SimplePublisher.java:389) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:244) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:223) at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:131) at java.base/sun.nio.ch.Invoker$3.run(Invoker.java:241) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)Exception in thread "AwsEventLoop 1" java.lang.OutOfMemoryError: Java heap spaceException in thread "AwsEventLoop 1" java.lang.OutOfMemoryError: Java heap space
2025-08-08 06:53:33,684 WARN s.a.a.s.s.i.c.S3CrtResponseHandlerAdapter [AwsEventLoop 1] Exception thrown in responsePublisher#error, ignoringjava.util.concurrent.CancellationException: subscription has been cancelled. at software.amazon.awssdk.utils.async.SimplePublisher.lambda$doProcessQueue$9(SimplePublisher.java:286) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.get(SimplePublisher.java:413) at software.amazon.awssdk.utils.async.SimplePublisher$FailureMessage.access$700(SimplePublisher.java:397) at software.amazon.awssdk.utils.async.SimplePublisher.doProcessQueue(SimplePublisher.java:260) at software.amazon.awssdk.utils.async.SimplePublisher.processEventQueue(SimplePublisher.java:224) at software.amazon.awssdk.utils.async.SimplePublisher.access$1300(SimplePublisher.java:58) at software.amazon.awssdk.utils.async.SimplePublisher$SubscriptionImpl.cancel(SimplePublisher.java:389) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.async.listener.SubscriberListener$NotifyingSubscriber$NotifyingSubscription.cancel(SubscriberListener.java:124) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:244) at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformer$FileSubscriber$1.failed(FileAsyncResponseTransformer.java:223) at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:131) at java.base/sun.nio.ch.Invoker$3.run(Invoker.java:241) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840)
2025-08-08 06:53:56,242 INFO o.o.m.b.c.S3Repo [main] Blob file download(s) complete
2025-08-08 06:53:56,243 ERROR o.o.m.b.c.S3Repo [main] FailedFileDownload(request=DownloadFileRequest(destination=/storage/s3_files/indices/PYO6TRrfSyuReHLFwBDXQw/3/__iLfGFpKiQD2AXo5P1ByBCQ, getObjectRequest=GetObjectRequest(Bucket=migration-artifacts-REDACTED-transforms-us-east-1, Key=newsnap/indices/PYO6TRrfSyuReHLFwBDXQw/3/__iLfGFpKiQD2AXo5P1ByBCQ)), exception=software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: OutOfMemoryError has been raised from JVM. (SDK Attempt Count: 1))
2025-08-08 06:53:56,244 ERROR o.o.m.b.c.S3Repo [main] FailedFileDownload(request=DownloadFileRequest(destination=/storage/s3_files/indices/PYO6TRrfSyuReHLFwBDXQw/3/__ZcBNnD9NRQGU7ecOFluSng, getObjectRequest=GetObjectRequest(Bucket=migration-artifacts-REDACTED-transforms-us-east-1, Key=newsnap/indices/PYO6TRrfSyuReHLFwBDXQw/3/__ZcBNnD9NRQGU7ecOFluSng)), exception=software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: OutOfMemoryError has been raised from JVM. (SDK Attempt Count: 1))
2025-08-08 06:53:56,244 ERROR o.o.m.b.c.S3Repo [main] FailedFileDownload(request=DownloadFileRequest(destination=/storage/s3_files/indices/PYO6TRrfSyuReHLFwBDXQw/3/__h7L4rQa1TKaLNPweeF8hRg, getObjectRequest=GetObjectRequest(Bucket=migration-artifacts-REDACTED-transforms-us-east-1, Key=newsnap/indices/PYO6TRrfSyuReHLFwBDXQw/3/__h7L4rQa1TKaLNPweeF8hRg)), exception=software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: OutOfMemoryError has been raised from JVM. (SDK Attempt Count: 1))
2025-08-08 06:53:56,244 ERROR o.o.m.b.c.S3Repo [main] FailedFileDownload(request=DownloadFileRequest(destination=/storage/s3_files/indices/PYO6TRrfSyuReHLFwBDXQw/3/__jq57CPb0SL2_VGJUpyUdkQ, getObjectRequest=GetObjectRequest(Bucket=migration-artifacts-REDACTED-transforms-us-east-1, Key=newsnap/indices/PYO6TRrfSyuReHLFwBDXQw/3/__jq57CPb0SL2_VGJUpyUdkQ)), exception=software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: OutOfMemoryError has been raised from JVM. (SDK Attempt Count: 1))
2025-08-08 06:53:56,244 ERROR o.o.m.b.c.S3Repo [main] FailedFileDownload(request=DownloadFileRequest(destination=/storage/s3_files/indices/PYO6TRrfSyuReHLFwBDXQw/3/__lXRI75L0TySLJ1tsAPwrtw, getObjectRequest=GetObjectRequest(Bucket=migration-artifacts-REDACTED-transforms-us-east-1, Key=newsnap/indices/PYO6TRrfSyuReHLFwBDXQw/3/__lXRI75L0TySLJ1tsAPwrtw)), exception=software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: subscription has been cancelled. (SDK Attempt Count: 1))
2025-08-08 06:53:56,244 ERROR o.o.m.b.c.S3Repo [main] FailedFileDownload(request=DownloadFileRequest(destination=/storage/s3_files/indices/PYO6TRrfSyuReHLFwBDXQw/3/__pcK2GVEfQ-63-LZKHVvB6w, getObjectRequest=GetObjectRequest(Bucket=migration-artifacts-REDACTED-transforms-us-east-1, Key=newsnap/indices/PYO6TRrfSyuReHLFwBDXQw/3/__pcK2GVEfQ-63-LZKHVvB6w)), exception=software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: OutOfMemoryError has been raised from JVM. (SDK Attempt Count: 1))
```

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

No OutOfMemoryError, Easy to configure and understand max on and off heap memory usage of Transfer Manager and CRT Client

### Current Behavior

See above

### Reproduction Steps

Was not able to create self-contained, concise snippet of code that can be used to reproduce the issue.

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### AWS Java SDK version used

2.32.4

### JDK version used

corretto 17

### Operating System and version

Amazon linux 2023 (ARM64)

Beginne mit S3TransferManager#downloadDirectory und der im Bericht beschriebenen S3AsyncClient.crtBuilder-Konfiguration. Reproduziere nach Möglichkeit die wiederholten Downloads großer Verzeichnisse und untersuche anschließend die im Stacktrace gezeigten Pfade von FileAsyncResponseTransformer und S3CrtResponseHandlerAdapter. Als erledigt gilt die Aufgabe, wenn die Ursache für die Erschöpfung des Java heaps identifiziert und eine Behebung oder eine zuverlässige Reproduktion ohne OutOfMemoryError validiert wurde.

Vom Indexierungsmodell aus dem Issue-Text verfasst.