aws / aws/aws-sdk-java-v2

S3: getObject, ResponseInputStream close not working as expected.

Aperta
#2,117 6 commenti 3 reazioni 0 assegnatari Vedi su GitHub
bug p3
Lingua principale
Java
Stelle
2.6k
Fork
1k
Merge medio
2g 9h
PR unite (30g)
51

Descrizione

## Describe the bug
When you invoke close on ResponseInputStream from the getObject before you reach the end of the data. The stream will not immediately(/within reasonable time) close on big files.

## Expected Behavior
When invoking close on an ResponseInputStream you would expect a close within at least a few seconds.

## Current Behavior
It does not close until it did a complete read of the stream internally. (this seems related to connection recycling internally).

## Steps to Reproduce
- Upload a big file to S3 atleast a 1-2 gb to observe the issue on fast internet connection
```
var r = client.getObject(c -> {
c.bucket(bucket).key(key).versionId(versionId);
});
r.read(new byte[1024]); // just to make sure we actually opened the resource and did something.
r.close(); // now you have to wait until the entire file is read by the close internally.
```

## Possible Solution
- The internal handler closes the stream immediately, possibly by simply closing the underlying connection.
- It could make a decision based on the remaining bytes to read to make the impact less significant on big files (e.g. if remaining > 1 MB close connection, otherwise consume).

## Context
I was trying to create a test case for a wrapper i created for handling network related issues (e.g. connection reset). Where i would simply track the position in the stream and use a range request when the connection fails, to continue the stream transparently.
When testing it by creating a wrapper around the ResponseInputStream that throws randomly a IO exception i noticed the extreme delay when closing the ResponseInputStream.

## Workaround
```
var r = client.getObject(c -> {
c.bucket(bucket).key(key).versionId(versionId);
});
r.read(new byte[1024]); // just to make sure we actually opened the resource and did something.
r.abort(); // <<< Workaround, but this is not part of a InputStream contract
r.close(); // now the connection closes almost immediately.
```
## Your Environment

* AWS Java SDK version used: 2.15.14
* JDK version used: 11
* Operating System and version: windows 10

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia con il percorso S3 getObject e il comportamento di chiusura e interruzione di ResponseInputStream descritto nell’issue. Riproduci il problema con un oggetto S3 di grandi dimensioni leggendo un buffer piccolo e chiamando close prima di raggiungere la fine, quindi traccia il comportamento interno di riciclo della connessione. L’attività è completata quando close restituisce il controllo entro pochi secondi ragionevoli senza richiedere abort né la lettura dell’oggetto rimanente.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, java
Ambito
api, cloud
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.