S3: getObject, ResponseInputStream close not working as expected.
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 51
Description
## Describe the bug
When you invoke close on ResponseInputStream from the getObject before you reach the end of the data. The stream will not immediately(/within reasonable time) close on big files.
## Expected Behavior
When invoking close on an ResponseInputStream you would expect a close within at least a few seconds.
## Current Behavior
It does not close until it did a complete read of the stream internally. (this seems related to connection recycling internally).
## Steps to Reproduce
- Upload a big file to S3 atleast a 1-2 gb to observe the issue on fast internet connection
```
var r = client.getObject(c -> {
c.bucket(bucket).key(key).versionId(versionId);
});
r.read(new byte[1024]); // just to make sure we actually opened the resource and did something.
r.close(); // now you have to wait until the entire file is read by the close internally.
```
## Possible Solution
- The internal handler closes the stream immediately, possibly by simply closing the underlying connection.
- It could make a decision based on the remaining bytes to read to make the impact less significant on big files (e.g. if remaining > 1 MB close connection, otherwise consume).
## Context
I was trying to create a test case for a wrapper i created for handling network related issues (e.g. connection reset). Where i would simply track the position in the stream and use a range request when the connection fails, to continue the stream transparently.
When testing it by creating a wrapper around the ResponseInputStream that throws randomly a IO exception i noticed the extreme delay when closing the ResponseInputStream.
## Workaround
```
var r = client.getObject(c -> {
c.bucket(bucket).key(key).versionId(versionId);
});
r.read(new byte[1024]); // just to make sure we actually opened the resource and did something.
r.abort(); // <<< Workaround, but this is not part of a InputStream contract
r.close(); // now the connection closes almost immediately.
```
## Your Environment
* AWS Java SDK version used: 2.15.14
* JDK version used: 11
* Operating System and version: windows 10
Contributor guide
Assessment
This issue has not been assessed yet.