aws / aws/aws-sdk-java-v2

ParallelMultipartDownloaderSubscriber.onError cancels part futures before completing resultFuture, swallowing the original error

Aperta Adatta ai principianti
#7,174 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug needs-triage
Lingua principale
Java
Stelle
2.6k
Fork
1k
Merge medio
2g 9h
PR unite (30g)
51

Descrizione

### Describe the bug

When a multipart download (Netty-based `S3AsyncClient` with `multipartEnabled(true)`, via `S3TransferManager.downloadFile`) fails, the caller's `completionFuture()` completes with a bare `java.util.concurrent.CancellationException` that has no cause attached. The actual `Throwable` that triggered the
failure is unrecoverable by the application.

Root cause is the ordering in `ParallelMultipartDownloaderSubscriber.onError` (`services/s3/src/main/java/software/amazon/awssdk/services/s3/internal/multipart/ParallelMultipartDownloaderSubscriber.java`):

@Override
public void onError(Throwable t) {
inFlightRequests.values().forEach(future -> future.cancel(true)); // 1. cancel first
inFlightRequests.clear();
resultFuture.completeExceptionally(t); // 2. real cause last
}

The `future.cancel(true)` calls propagate a `CancellationException` through the transformer chain (`FileAsyncResponseTransformerPublisher`) to the future the caller observes, before `resultFuture.completeExceptionally(t)` runs. The method also does not log `t`, so the trigger is invisible even at
DEBUG level.

Related earlier report: #6612 (closed for staleness without a fix).

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

The caller-visible future should fail with the original `Throwable` that triggered `onError` (or at minimum a `CancellationException` whose cause is `t`), and `t` should be logged, so applications can classify and handle the real failure.

### Current Behavior

The caller's `completionFuture()` fails with a bare `CancellationException` with no cause and no suppressed exceptions. Stack trace observed in production:

java.util.concurrent.CancellationException
at java.base/java.util.concurrent.CompletableFuture.cancel(CompletableFuture.java:2478)
at software.amazon.awssdk.services.s3.internal.multipart.ParallelMultipartDownloaderSubscriber.lambda$onError$18(ParallelMultipartDownloaderSubscriber.java:418)
at java.base/java.util.concurrent.ConcurrentHashMap$ValuesView.forEach(ConcurrentHashMap.java:4780)
at software.amazon.awssdk.services.s3.internal.multipart.ParallelMultipartDownloaderSubscriber.onError(ParallelMultipartDownloaderSubscriber.java:418)
at software.amazon.awssdk.utils.internal.MappingSubscriber.onError(MappingSubscriber.java:60)
at software.amazon.awssdk.core.internal.async.FileAsyncResponseTransformerPublisher$IndividualFileTransformer.onResponse(FileAsyncResponseTransformerPublisher.java:111)
at software.amazon.awssdk.core.async.listener.AsyncResponseTransformerListener$NotifyingAsyncResponseTransformer.onResponse(AsyncResponseTransformerListener.java:92)
at software.amazon.awssdk.core.internal.http.async.AsyncStreamingResponseHandler.onHeaders(AsyncStreamingResponseHandler.java:55)
at software.amazon.awssdk.http.nio.netty.internal.ResponseHandler.channelRead0(ResponseHandler.java:101)
...

Note there is no `Caused by:` — the original error that triggered `onError` is lost. No log line is emitted by the subscriber either, so the trigger cannot be recovered even with SDK DEBUG logging enabled.

### Reproduction Steps

Any failure injected into an in-flight multipart download reproduces the cause-swallowing. The simplest deterministic repro is to close the client while a large download is in flight (the terminated scheduled executor rejects a part retry, which triggers `onError`, but any part-level failure takes
the same path):

S3AsyncClient s3 = S3AsyncClient.builder()
.region(Region.US_EAST_1)
.multipartEnabled(true)
.build();
S3TransferManager tm = S3TransferManager.builder().s3Client(s3).build();

FileDownload download = tm.downloadFile(DownloadFileRequest.builder()
.getObjectRequest(b -> b.bucket("").key(""))
.destination(Paths.get("/tmp/out.bin"))
.build());

// Induce a failure mid-download, e.g. close the client while parts are in flight
Thread.sleep(500);
tm.close();
s3.close();

try {
download.completionFuture().join();
} catch (CompletionException e) {
Throwable cause = e.getCause();
System.out.println(cause); // java.util.concurrent.CancellationException
System.out.println(cause.getCause()); // null <-- original trigger lost
}

Expected: the future fails with the underlying error (here a `RejectedExecutionException` from the terminated executor), or a `CancellationException` carrying it as cause. Actual: a bare `CancellationException`, cause `null`.

### Possible Solution

This is already fixed in the sibling class in the same package. `ParallelPresignedUrlMultipartDownloaderSubscriber.onError` completes `resultFuture` before cancelling, and logs the error:

@Override
public void onError(Throwable t) {
log.debug(() -> "Error in parallel multipart download", t);
resultFuture.completeExceptionally(t);
inFlightRequests.values().forEach(future -> future.cancel(true));
}

Its `resultFuture` field Javadoc documents the reasoning: "Completed exceptionally on error (before cancel)...". Applying the same ordering and logging to `ParallelMultipartDownloaderSubscriber` resolves this:

@Override
public void onError(Throwable t) {
log.debug(() -> "Error in parallel multipart download", t);
resultFuture.completeExceptionally(t);
inFlightRequests.values().forEach(future -> future.cancel(true));
inFlightRequests.clear();
}

### Additional Information/Context

Observed in a production service performing concurrent large-file multipart downloads. Failures arrive in clusters; because no cause survives, the application cannot distinguish transient client-side conditions from real S3 errors, forcing misclassification. We separately confirmed via a
request-level interceptor that at least one trigger cohort is a `RejectedExecutionException` from a terminated scheduled executor, but that evidence is only available at the HTTP layer — the transfer-level future discards it.

### AWS Java SDK version used

AWS Java SDK version: 2.x (Netty async client; bug present in current master per cited source)

### JDK version used

JDK: 17

### Operating System and version

Amazon Linux 2 (x86_64)

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia da services/s3/src/main/java/software/amazon/awssdk/services/s3/internal/multipart/ParallelMultipartDownloaderSubscriber.java, concentrandoti su onError e sull’ordine di resultFuture. Confrontalo con ParallelPresignedUrlMultipartDownloaderSubscriber.onError, quindi verifica che un errore di multipart-download iniettato preservi il Throwable originale ed emetta il log di debug descritto.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, java
Ambito
backend, cloud
Tipo di issue
Bug
Difficoltà
2/5
Tempo stimato
1-3 ore
Stato di attività
Tranquilla
Chiarezza
Specificata chiaramente
Idoneità per principianti
76/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.