GoogleContainerTools / GoogleContainerTools/jib

Incorrect progress report usage in BlobPuller

Open
#1,522 0 comments 0 reactions 0 assignees View on GitHub
area/jib-core cleanup priority: p2
Dominant language
Java
Stars
14.5k
Forks
1.5k
PR merge metrics
No merged PRs in 30d

Description

From https://github.com/GoogleContainerTools/jib/issues/1512#issuecomment-468078006.

```java
public Void handleResponse(Response response) ... {
// Notifies the progress dispatcher that the total bytes we will read is the "Content-Length" value.
// If `Content-Encoding: gzip`, the value will be the size of the compressed content.
blobSizeConsumer.accept(response.getContentLength());

try (OutputStream outputStream =
new NotifyingOutputStream(destinationOutputStream, writtenByteCountListener)) {
BlobDescriptor receivedBlobDescriptor =
Digests.computeDigest(response.getBody(), outputStream);
...
}
```
Now, in the case of `Content-Encoding: gzip` (or something similar), Google HTTP Client's `HttpResponse::getContent()` creates and returns `GZIPInputStream` that wraps the raw `InputStream`, which means the library does the chore of streamed unzipping on behalf of us. `Content-Length` will of course be the size of the original (compressed) content stream.

Then `NotifyingOutputStream::write()` will periodically notify the progress dispatcher how many bytes were written (through `writtenByteCountListener`). The problem is that the progress dispatcher is initialized with the total allocation size (through `blobSizeConsumer`) to be the value reported in `Content-Length`. Because we will write uncompressed content to `outputStream` in this case, we will go over the progress limit.

---

Another case where the progress is under reported: https://github.com/GoogleContainerTools/jib/blob/85bb1c58f451a1949f35c8a34bc381c40f8fe047/jib-core/src/main/java/com/google/cloud/tools/jib/builder/steps/ExtractTarStep.java#L155

The `NotifyingOutputStream` will report written byte count. The count will be smaller than the original file size, as the written content is compressed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.