S3: getObject, ResponseInputStream close not working as expected.
- Lingua principale
- Java
- Stelle
- 2.6k
- Fork
- 1k
- Merge medio
- 2g 9h
- PR unite (30g)
- 51
Descrizione
## Describe the bug
When you invoke close on ResponseInputStream from the getObject before you reach the end of the data. The stream will not immediately(/within reasonable time) close on big files.
## Expected Behavior
When invoking close on an ResponseInputStream you would expect a close within at least a few seconds.
## Current Behavior
It does not close until it did a complete read of the stream internally. (this seems related to connection recycling internally).
## Steps to Reproduce
- Upload a big file to S3 atleast a 1-2 gb to observe the issue on fast internet connection
```
var r = client.getObject(c -> {
c.bucket(bucket).key(key).versionId(versionId);
});
r.read(new byte[1024]); // just to make sure we actually opened the resource and did something.
r.close(); // now you have to wait until the entire file is read by the close internally.
```
## Possible Solution
- The internal handler closes the stream immediately, possibly by simply closing the underlying connection.
- It could make a decision based on the remaining bytes to read to make the impact less significant on big files (e.g. if remaining > 1 MB close connection, otherwise consume).
## Context
I was trying to create a test case for a wrapper i created for handling network related issues (e.g. connection reset). Where i would simply track the position in the stream and use a range request when the connection fails, to continue the stream transparently.
When testing it by creating a wrapper around the ResponseInputStream that throws randomly a IO exception i noticed the extreme delay when closing the ResponseInputStream.
## Workaround
```
var r = client.getObject(c -> {
c.bucket(bucket).key(key).versionId(versionId);
});
r.read(new byte[1024]); // just to make sure we actually opened the resource and did something.
r.abort(); // <<< Workaround, but this is not part of a InputStream contract
r.close(); // now the connection closes almost immediately.
```
## Your Environment
* AWS Java SDK version used: 2.15.14
* JDK version used: 11
* Operating System and version: windows 10
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia con il percorso S3 getObject e il comportamento di chiusura e interruzione di ResponseInputStream descritto nell’issue. Riproduci il problema con un oggetto S3 di grandi dimensioni leggendo un buffer piccolo e chiamando close prima di raggiungere la fine, quindi traccia il comportamento interno di riciclo della connessione. L’attività è completata quando close restituisce il controllo entro pochi secondi ragionevoli senza richiedere abort né la lettura dell’oggetto rimanente.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- aws, java
- Ambito
- api, cloud
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100