google / google/go-containerregistry
crane: registry needs to hold full layers in memory
- Dominant language
- Go
- Stars
- 4k
- Forks
- 686
- Avg merge
- 2d 12h
- Merged PRs (30d)
- 26
Description
### Describe the bug
When using `crane registry serve` and pushing large layers with `PUT`, I observe excessive memory usage.
I traced this to the `PUT` handler function:
https://github.com/google/go-containerregistry/blob/59a4b85930392a30c39462519adc8a2026d47181/pkg/registry/blobs.go#L485
The in-flight blob is cached in full in a buffer until all chunks have been uploaded, and then `Put` is invoked after the transfer is complete.
This creates two problems:
- Excessive memory usage: with a typical CUDA base image, I can reliably produce OOM situations, since individual layers are gigabytes in size.
- The CLI side looks broken to the end-user: after every byte was sent via HTTP, the progress bar is at 100%, but the server still has to call `Put`. For a slow `BlobHandler` (one that has to upload data to a remote storage backend), this results in minutes of waiting where the progress bar is full and the user thinks that the upload may be stuck.
### To Reproduce
Terminal 1:
```
crane registry serve --address 127.0.0.1:1234
```
```
crane copy nvidia/cuda@sha256:f353ffca86e0cd93ab2470fe274ecf766519c24c37ed58cc2f91d915f7ebe53c localhost:1234/nvidia/cuda:latest
```
### Expected behavior
I want the memory usage to be reasonable, even for large blobs.
### Additional context
I'd like to provide a fix in the form of a custom `io.Reader` with an internal fixed-size buffer that allows a `BlobHandler` to read data in chunks.
This should make uploads faster, less memory intensive, and provide better feedback to users.
Is this an acceptable solution?
Contributor guide
Research direction
Start with the PUT handler in pkg/registry/blobs.go around line 485 and inspect how BlobHandler receives completed uploads. Run crane registry serve with the provided crane copy reproduction, then verify large blob uploads no longer require the entire layer in memory and that the handler's post-upload delay is addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100