aws / aws/amazon-s3-encryption-client-java

Memory leak in S3EncryptionClient.getObject()

Aperta Adatta ai principianti
#518 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Java
Stelle
34
Fork
21
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

### Security issue notifications

If you discover a potential security issue in the Amazon S3 Encryption Client we ask that you notify AWS Security via our [vulnerability reporting page](https://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public GitHub issue.

_noted_ ✅

### Problem:

S3EncryptionClient.getObject() leaks memory on every call because the ResponseInputStream returned by the internal async pipeline is never closed.

Code in question: `src/main/java/software/amazon/encryption/s3/S3EncryptionClient.java`

```
@Override
public T getObject(GetObjectRequest getObjectRequest,
ResponseTransformer responseTransformer)
throws AwsServiceException, SdkClientException {

// GetEncryptedObjectPipeline builder

try {
ResponseInputStream joinFutureGet = pipeline.getObject(getObjectRequest, AsyncResponseTransformer.toBlockingInputStream()).join();
return responseTransformer.transform(joinFutureGet.response(), AbortableInputStream.create(joinFutureGet));
} catch (CompletionException e) {
throw new S3EncryptionClientException(e.getCause().getMessage(), e.getCause());
} catch (Exception e) {
throw new S3EncryptionClientException("Unable to transform response.", e);
}
}
```

The `joinFutureGet` ResponseInputStream is never closed so in an internal service where we use this method to read client side encrypted files in batches we get these errors:

```
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 2048 byte(s) of direct memory (used: 7,025,458,950, max: 7,025,459,200)
at io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:97)
at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:168)
at io.netty.handler.ssl.SslHandler.allocate(SslHandler.java:2381)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1488)
```

### Reproduction test:

```
import java.lang.management.BufferPoolMXBean;
import java.lang.management.ManagementFactory;
import java.net.URI;
import software.amazon.awssdk.auth.credentials.AwsBasicCredentials;
import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.S3Configuration;
import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.encryption.s3.S3EncryptionClient;

// Requires LocalStack: docker run -d -p 4566:4566 -e SERVICES=s3 localstack/localstack:3.0.2

var creds = StaticCredentialsProvider.create(AwsBasicCredentials.create("test", "test"));
var endpoint = URI.create("http://localhost:4566");

var encryptionClient = S3EncryptionClient.builder()
.credentialsProvider(creds)
.region(Region.EU_CENTRAL_1)
.endpointOverride(endpoint)
.forcePathStyle(true)
.kmsKeyId("alias/test-key")
.enableLegacyWrappingAlgorithms(true)
.enableLegacyUnauthenticatedModes(true)
.build();

// Create bucket and upload a test file
encryptionClient.createBucket(CreateBucketRequest.builder().bucket("test").build());
encryptionClient.putObject(
PutObjectRequest.builder().bucket("test").key("file.txt").build(),
RequestBody.fromString("hello"));

long before = getDirectMemoryUsed();

for (int i = 0; i < 5000; i++) {
try {
encryptionClient.getObjectAsBytes(
GetObjectRequest.builder().bucket("test").key("file.txt").build());
} catch (Exception e) {
// Decryption fails (no real KMS), but the leak happens regardless
}
}

long after = getDirectMemoryUsed();
System.out.printf("Leaked: %,d bytes in 5000 calls (~%d bytes/call)%n", after - before, (after - before) / 5000);

static long getDirectMemoryUsed() {
return ManagementFactory.getPlatformMXBeans(BufferPoolMXBean.class).stream()
.filter(pool -> pool.getName().equals("direct"))
.mapToLong(BufferPoolMXBean::getMemoryUsed)
.sum();
}
```
Results:

5000 getObjectAsBytes() calls:
Direct memory before: 45,056 bytes
Direct memory after: 4,993,032 bytes
Leaked: 4,947,976 bytes

Also got a warning:
```
LEAK: ByteBuf.release() was not called before it's garbage-collected.
Created at:
io.netty.buffer.AdaptiveByteBufAllocator.newDirectBuffer(AdaptiveByteBufAllocator.java:67)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:168)
...
```

### Solution:

Open the stream with try catch with resources block eg:

```
try (ResponseInputStream joinFutureGet =
pipeline.getObject(getObjectRequest, AsyncResponseTransformer.toBlockingInputStream()).join()) {
return responseTransformer.transform(joinFutureGet.response(), AbortableInputStream.create(joinFutureGet));
} catch (CompletionException e) {
throw new S3EncryptionClientException(e.getCause().getMessage(), e.getCause());
} catch (Exception e) {
throw new S3EncryptionClientException("Unable to transform response.", e);
}
```

### Out of scope:

N/A

☑️ (NOTE: If you believe this might be a security issue, please email aws-security@amazon.com instead of creating a GitHub issue. For more details, see the AWS Vulnerability Reporting Guide: https://aws.amazon.com/security/vulnerability-reporting/ )

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Leggi src/main/java/software/amazon/encryption/s3/S3EncryptionClient.java, concentrandoti su getObject() e sul ResponseInputStream restituito dalla pipeline asincrona. Riproduci il problema con il test LocalStack fornito da 5,000 chiamate, quindi verifica che le chiamate ripetute non aumentino più la memoria diretta né producano avvisi di perdita di Netty.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, java
Ambito
api, backend, cloud
Tipo di issue
Bug
Difficoltà
2/5
Tempo stimato
1-3 ore
Stato di attività
Tranquilla
Chiarezza
Specificata chiaramente
Idoneità per principianti
82/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.