aws / aws/aws-sdk-java-v2

ResponseInputStream breaks GZIPInputStream on concatenated gzip streams (silent data truncation, v1 parity regression)

Aperta
#7,046 7 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug p2
Lingua principale
Java
Stelle
2.6k
Fork
1k
Merge medio
2g 9h
PR unite (30g)
51

Descrizione

**Related:** #4522 (closed for staleness — same underlying cause, never fixed)

## Description

When reading **concatenated gzip files** (multiple gzip members appended together, e.g., ALB access logs) from S3, wrapping the SDK v2 `ResponseInputStream` in `java.util.zip.GZIPInputStream` causes **silent data truncation**. The same code path worked correctly with SDK v1's `S3Object.getObjectContent()`.

This is a parity regression: migrating from v1 to v2 with no other code changes introduces non-deterministic data loss on concatenated gzip streams.

## Environment

- **AWS SDK v2 version:** 2.x (reproducible across recent versions)
- **Java version:** 11+
- **OS:** Linux (Amazon ECS / Fargate)

## Expected Behavior

Reading a concatenated gzip file from S3 through `GZIPInputStream` should return **all bytes from all gzip members**, identical to SDK v1 behavior.

A 318 MB concatenated gzip file (e.g., ALB logs with thousands of gzip members) should decompress fully regardless of buffer alignment.

## Actual Behavior

`GZIPInputStream` reads only a **partial, non-deterministic** amount of data — between 4 MB and 59 MB from a 318 MB file — and then returns `-1` (EOF), silently discarding the remaining gzip members.

The amount read varies between runs because the truncation depends on which internal buffer boundary a gzip member end happens to land on.

## Root Cause Analysis

`java.util.zip.GZIPInputStream` handles concatenated gzip streams by checking for additional members after each member ends. At member boundaries, it calls `in.available()` on the underlying `InflaterInputStream`, which in turn calls `in.available()` on the source stream. The logic in `GZIPInputStream` (see [OpenJDK source](https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/zip/GZIPInputStream.java)) interprets a `0` return from `available()` combined with a `-1` read as end-of-stream.

**SDK v1 (`S3ObjectInputStream`):** Wraps an Apache HTTP `InputStream` which implements `available()` in a way that `GZIPInputStream`'s concatenated-member detection works reliably. The underlying stream's buffering ensures bytes are typically "available" at member boundaries.

**SDK v2 (`ResponseInputStream`):** Uses a different HTTP client implementation (Netty-based or URL connection-based) where the `InputStream`'s `available()` semantics differ. At gzip member boundaries, when `GZIPInputStream` checks whether there's another member, the v2 stream can report `0` available (because the next chunk hasn't arrived from the network yet), causing `GZIPInputStream` to conclude the stream is exhausted.

This is timing-dependent: it occurs when a gzip member boundary aligns with an empty network buffer — which is common with large files over a network connection but rare in local/unit tests with `ByteArrayInputStream`.

## Reproduction

### Minimal setup

1. Create a concatenated gzip file (multiple gzip members appended):
```java
ByteArrayOutputStream baos = new ByteArrayOutputStream();
for (int i = 0; i < 1000; i++) {
GZIPOutputStream gzipOut = new GZIPOutputStream(baos);
gzipOut.write(("member " + i + "\n").getBytes());
gzipOut.finish();
gzipOut.flush();
}
// Upload baos.toByteArray() to S3
```

2. Read with SDK v2:
```java
S3Client s3 = S3Client.create();
ResponseInputStream ris = s3.getObject(
GetObjectRequest.builder().bucket("bucket").key("concatenated.gz").build()
);
BufferedInputStream bis = new BufferedInputStream(ris, 65536);
GZIPInputStream gzis = new GZIPInputStream(bis, 65536);

byte[] buf = new byte[8192];
long total = 0;
int n;
while ((n = gzis.read(buf)) != -1) {
total += n;
}
// 'total' will be LESS than the full decompressed size on large files
// The exact amount varies per run (non-deterministic)
```

**Note:** This is difficult to reproduce with small files or local streams because it requires the network read to return 0 bytes available at a gzip member boundary. Use a file with many (hundreds+) gzip members totaling >100 MB to reliably trigger the issue.

### Production evidence

Migrating an S3 reader from SDK v1 to v2 with no other changes:

| SDK | File size (compressed) | Bytes decompressed | Result |
|-----|----------------------|-------------------|--------|
| v1 (`AmazonS3.getObject()`) | 318 MB | 318,926,634 (full) | Correct |
| v2 (`S3Client.getObject()`) | 318 MB | 4–59 MB (varies) | **Truncated** |

## Workaround

Replace `java.util.zip.GZIPInputStream` with Apache Commons Compress `GzipCompressorInputStream(stream, true)`, which handles concatenated members without relying on `available()` semantics:

```java
import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;

ResponseInputStream ris = s3.getObject(request);
BufferedInputStream bis = new BufferedInputStream(ris, 65536);
GzipCompressorInputStream gzis = new GzipCompressorInputStream(bis, true);
// reads all concatenated members correctly
```

## Suggested Fix

The SDK v2 `ResponseInputStream` (or its underlying `AbortableInputStream`) should ensure that `available()` does not return `0` when there are unread bytes remaining in the stream, or at minimum should match the `available()` contract that SDK v1's stream provided.

Possible approaches:
1. **Override `available()`** in `ResponseInputStream` to return `1` when the stream is not exhausted (consistent with the `InputStream` contract which states `available()` returning `0` does not mean EOF, but `GZIPInputStream` uses it as a heuristic).
2. **Wrap the response stream in a `BufferedInputStream`** internally so that `available()` reflects buffered bytes, matching v1 behavior.
3. **Document the incompatibility** in the v1→v2 migration guide with explicit guidance to use `GzipCompressorInputStream` for concatenated gzip streams.

## Impact

Any application that:
- Reads concatenated gzip files from S3 (ALB logs, CloudFront logs, custom log pipelines)
- Uses `java.util.zip.GZIPInputStream`
- Migrated from SDK v1 to v2

...is silently losing data. The non-deterministic nature makes it difficult to detect — the stream appears to complete successfully, just with fewer bytes.

## References

- [OpenJDK GZIPInputStream concatenated member handling](https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/util/zip/GZIPInputStream.java)
- [RFC 1952 §2.2](https://datatracker.ietf.org/doc/html/rfc1952#section-2.2): "A gzip file consists of a series of 'members' (compressed data sets). [...] The members simply appear one after another in the file"
- [AWS SDK v1 S3ObjectInputStream](https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/model/S3ObjectInputStream.java)
- [AWS SDK v2 ResponseInputStream](https://github.com/aws/aws-sdk-java-v2/blob/master/core/sdk-core/src/main/java/software/amazon/awssdk/core/ResponseInputStream.java)

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia da core/sdk-core/src/main/java/software/amazon/awssdk/core/ResponseInputStream.java e dal comportamento collegato di OpenJDK GZIPInputStream; esegui la riproduzione S3 con membri concatenati di grandi dimensioni usando Java 11+ e un BufferedInputStream. Confronta available() e i conteggi dei byte decompressi ai confini dei membri gzip. Il lavoro è completato quando l’intero stream concatenato viene letto in modo coerente, con una copertura di regressione per il troncamento segnalato.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, java
Ambito
api, backend
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
55/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.