aws / aws/aws-sdk-java-v2

Validate S3 GetObject before reading -1 from the underlying input stream

Aperta
#5,422 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
feature-request needs-triage
Lingua principale
Java
Stelle
2.6k
Fork
1k
Merge medio
2g 9h
PR unite (30g)
51

Descrizione

### Describe the feature

Currently, [ChecksumValidatingInputStream](https://github.com/aws/aws-sdk-java-v2/blob/master/core/sdk-core/src/main/java/software/amazon/awssdk/core/internal/io/ChecksumValidatingInputStream.java) validates the checksum only when the `read()` from the underlying InputStream returns `-1`. If we know we are reading a full object and we know we have read the full stream's bytes, we could validate at that point instead of relying on 1 more `read()` that returns `-1`.

### Use Case

Some users may stream the bytes into a byte array that is exactly the size of the object. If that loop ends once the byte array size number of bytes have been read, validation would not happen. The validation would only happen if the user calls `InputStream.read()` one more time to receive a result of -1.

Users of guava's [ByteStreams.readFully()](https://guava.dev/releases/19.0/api/docs/com/google/common/io/ByteStreams.html#readFully(java.io.InputStream,%20byte[])) would also exhibit this behavior. See an example of how the loop is implemented [here](https://github.com/google/guava/blob/master/guava/src/com/google/common/io/ByteStreams.java#L925)

As a workaround, users could ensure they are doing checksum validation by reading until they receive `-1`. If current users don't dig into the validation code, they may not realize the SDK only does checksum validation on the read call after the last byte is read.

### Proposed Solution

If we know that the GetObjectRequest is for a full object, make the validation in ChecksumValidatingInputStream happen once the last byte is read instead of when the underlying InputStream returns `-1`

### Other Information

_No response_

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### AWS Java SDK version used

2.26.23

### JDK version used

17

### Operating System and version

Amazon Linux 2

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia da core/sdk-core/src/main/java/software/amazon/awssdk/core/internal/io/ChecksumValidatingInputStream.java, quindi segui il percorso degli stream di GetObjectRequest per oggetti completi fino a raggiungerlo. Riproduci i casi descritti nell’issue con array di byte di dimensione esatta e Guava ByteStreams.readFully(), quindi verifica che la lettura di un oggetto completo convalidi il checksum senza richiedere una lettura aggiuntiva che restituisca -1.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, java
Ambito
cloud
Tipo di issue
Funzionalità
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.