aws / aws/amazon-s3-encryption-client-java
OOM and request cancellation when reading large objects
- 主要言語
- Java
- スター
- 34
- フォーク
- 21
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
### Problem:
When attempting to read a large file (~1GB) from S3 using the v3 Encryption SDK, the request fails with a hidden OutOfMemoryError. In v2 SDK, the same operation works without issues.
Stack trace:
```
java.io.IOException: Request cancelled
at software.amazon.awssdk.http.nio.netty.internal.FutureCancelHandler.exceptionCaught(FutureCancelHandler.java:63)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:424)
at io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:92)
at io.netty.channel.CombinedChannelDuplexHandler$1.fireExceptionCaught(CombinedChannelDuplexHandler.java:145)
at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
at io.netty.channel.CombinedChannelDuplexHandler.exceptionCaught(CombinedChannelDuplexHandler.java:231)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.handler.ssl.SslHandler.exceptionCaught(SslHandler.java:1221)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1324)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:856)
at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.lambda$createExecutionFuture$3(NettyRequestExecutor.java:135)
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:106)
at io.netty.util.concurrent.AbstractEventExecutor.runTask$$$capture(AbstractEventExecutor.java:173)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:166)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:998)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at java.base/java.lang.Thread.run(Thread.java:840)
```
The exception reported is misleading (FutureCancelledException), but after debugging I found it’s masking an actual OOM:
```
software.amazon.awssdk.http.nio.netty.internal.FutureCancelledException: java.lang.OutOfMemoryError: Java heap space
```
Steps to reproduce:
```java
S3EncryptionClient s3EncryptionClient = S3EncryptionClient.builder()
.enableDelayedAuthenticationMode(true)
.kmsKeyId("XXX")
.build();
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket("XXX")
.key("XXX")
.build();
try (BufferedReader stream = new BufferedReader(new InputStreamReader(
s3EncryptionClient.getObject(getObjectRequest, ResponseTransformer.toInputStream())))) {
String line;
while ((line = stream.readLine()) != null) {
System.out.println("Read line: " + line);
// Process the line
}
}
```
I am using the latest version 3.4.0. The file size being tested is approximately 1GB. The runtime environment is Java 17.
コントリビューションガイド
調査の方向性
Start with the S3EncryptionClient.getObject call using ResponseTransformer.toInputStream and reproduce the 1GB read on Java 17 with delayed authentication enabled. Trace where the hidden OutOfMemoryError becomes FutureCancelledException; done means large-object reads no longer exhaust the heap and the reported failure is accurate, with a regression test for the reproduction.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- aws, java
- 領域
- backend, security
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100