Discussion: "Diff" Pulls?

No files, tests, or entry points are named. Start by reviewing the distribution specification and the three-comment discussion around image pulls and manifests. Done means reaching a clear decision on whether clients can advertise an older image and receive an immutable server-provided diff with manifest-hash verification and full-pull fallback.

Should we consider "image diffs" as part of the spec?

For most clients pulling an image, they have a very recent, slightly older version of the image. Some clients, such as those in "edge" deployments, may have very constrained WAN bandwidth where it's important to maximize transmission efficiency. In most CI/CD setups, the actual binary differences between one image version and the next may be very small relative to the overall image size.

Consequently it would seem advantageous for:

1. A client to be able to advertise that it already has a given, older version when performing a pull request.
2. The server, if in possession of the older AND newer version, to be able to vend a diff to the client.

After the client receives and applies the diff, the client could use the hash from the manifest to verify that the "patched" image is indeed identical to the new image. If a discrepancy is noted (e.g. because the diff was corrupted or something went wrong with the patching process), the client could log the error and fall back to the current method of just doing a full pull of the new image.

Note that a given diff from Version $X to Version $Y is cacheable and immutable.

Tools like bsdiff & bspatch or similar could be used to actually perform the image diffing and patching.

Questions:

1. Does the problem formulation make sense?
2. While thoughtful/custom packaging of stacked image layers could allow a clever builder & client to effectively perform this function without needing to modify the repository protocol or implementation, this would severely constrain the set of containers for which this efficient distribution could be deployed. So the hope would be to have a solution that most clients could use with most containers and most repositories to be able to efficiently update their images.