GoogleContainerTools / GoogleContainerTools/jib
Incorrect progress report usage in BlobPuller
- Dominant language
- Java
- Stars
- 14.5k
- Forks
- 1.5k
- PR merge metrics
- No merged PRs in 30d
Description
From https://github.com/GoogleContainerTools/jib/issues/1512#issuecomment-468078006.
```java
public Void handleResponse(Response response) ... {
// Notifies the progress dispatcher that the total bytes we will read is the "Content-Length" value.
// If `Content-Encoding: gzip`, the value will be the size of the compressed content.
blobSizeConsumer.accept(response.getContentLength());
try (OutputStream outputStream =
new NotifyingOutputStream(destinationOutputStream, writtenByteCountListener)) {
BlobDescriptor receivedBlobDescriptor =
Digests.computeDigest(response.getBody(), outputStream);
...
}
```
Now, in the case of `Content-Encoding: gzip` (or something similar), Google HTTP Client's `HttpResponse::getContent()` creates and returns `GZIPInputStream` that wraps the raw `InputStream`, which means the library does the chore of streamed unzipping on behalf of us. `Content-Length` will of course be the size of the original (compressed) content stream.
Then `NotifyingOutputStream::write()` will periodically notify the progress dispatcher how many bytes were written (through `writtenByteCountListener`). The problem is that the progress dispatcher is initialized with the total allocation size (through `blobSizeConsumer`) to be the value reported in `Content-Length`. Because we will write uncompressed content to `outputStream` in this case, we will go over the progress limit.
---
Another case where the progress is under reported: https://github.com/GoogleContainerTools/jib/blob/85bb1c58f451a1949f35c8a34bc381c40f8fe047/jib-core/src/main/java/com/google/cloud/tools/jib/builder/steps/ExtractTarStep.java#L155
The `NotifyingOutputStream` will report written byte count. The count will be smaller than the original file size, as the written content is compressed.
Contributor guide
Assessment
This issue has not been assessed yet.