Validate S3 GetObject before reading -1 from the underlying input stream
- Lingua principale
- Java
- Stelle
- 2.6k
- Fork
- 1k
- Merge medio
- 2g 9h
- PR unite (30g)
- 51
Descrizione
### Describe the feature
Currently, [ChecksumValidatingInputStream](https://github.com/aws/aws-sdk-java-v2/blob/master/core/sdk-core/src/main/java/software/amazon/awssdk/core/internal/io/ChecksumValidatingInputStream.java) validates the checksum only when the `read()` from the underlying InputStream returns `-1`. If we know we are reading a full object and we know we have read the full stream's bytes, we could validate at that point instead of relying on 1 more `read()` that returns `-1`.
### Use Case
Some users may stream the bytes into a byte array that is exactly the size of the object. If that loop ends once the byte array size number of bytes have been read, validation would not happen. The validation would only happen if the user calls `InputStream.read()` one more time to receive a result of -1.
Users of guava's [ByteStreams.readFully()](https://guava.dev/releases/19.0/api/docs/com/google/common/io/ByteStreams.html#readFully(java.io.InputStream,%20byte[])) would also exhibit this behavior. See an example of how the loop is implemented [here](https://github.com/google/guava/blob/master/guava/src/com/google/common/io/ByteStreams.java#L925)
As a workaround, users could ensure they are doing checksum validation by reading until they receive `-1`. If current users don't dig into the validation code, they may not realize the SDK only does checksum validation on the read call after the last byte is read.
### Proposed Solution
If we know that the GetObjectRequest is for a full object, make the validation in ChecksumValidatingInputStream happen once the last byte is read instead of when the underlying InputStream returns `-1`
### Other Information
_No response_
### Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### AWS Java SDK version used
2.26.23
### JDK version used
17
### Operating System and version
Amazon Linux 2
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia da core/sdk-core/src/main/java/software/amazon/awssdk/core/internal/io/ChecksumValidatingInputStream.java, quindi segui il percorso degli stream di GetObjectRequest per oggetti completi fino a raggiungerlo. Riproduci i casi descritti nell’issue con array di byte di dimensione esatta e Guava ByteStreams.readFully(), quindi verifica che la lettura di un oggetto completo convalidi il checksum senza richiedere una lettura aggiuntiva che restituisca -1.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- aws, java
- Ambito
- cloud
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100