Measure compressionCache effectiveness; if it does not pay, revert to single-pass layer writes

compressionCache (pkg/build/build.go:53) is an unbounded process-global sync.Map of diffID → *v1.Descriptor. It exists so Digest() and Size() can answer without recompressing a layer whose content was already compressed in this process.

It is also the reason layer writes are two-pass. db97d1ee ("Lazily compress layers, maintain diffID -> digest cache", #1735) introduced the laziness and the cache in the same commit. Before it, newLayerWriter was:

gzw := pooledGzipWriter(io.MultiWriter(digest, buf))
w := tar.NewWriter(io.MultiWriter(diffid, gzw))

Deferring compression requires a materialized plain tar to defer against, so the current path writes the layer twice: plain tar first, then a second gzipped copy beside it.

The question

Is the cache earning that? The stated rationale on #1735 was that a cached diffID → digest lets a registry HEAD return 200 so we skip compression entirely. That only pays at a meaningful hit rate, and the hit rate has never been measured — there is no instrumentation on Store/Load today.

One data point suggesting the ceiling is low: benchmarked at a 100% hit rate (three identical 512 MiB layers, so the second and third skip compression outright), the two-pass path was still slower than a single-pass write — 1.476s vs 1.284s. The cache can only skip the compression; the plain-tar write happens either way and is the larger cost. If that generalizes, hit rate does not rescue it.

Proposed work

1. Instrument compressionCache hit/miss in Digest() and Size(). A local approximation is also possible: build a representative set of images in one process with a layering strategy and count repeated diffIDs.
2. If the hit rate is low — or high but still dominated by the write — revert #1735's write path. Single-pass becomes the only mode, which deletes layer.compress(), the cache, and the lazy descriptor machinery.
3. If it does pay, document why, and bound the map. It currently has no eviction, so it grows for the life of the process.

Relationship to #2479

#2479 adds single-pass writing behind an opt-in option and does not touch the cache; the default path and the cache are unchanged there. This issue is the larger follow-up: if the measurement supports it, the option added by #2479 goes away along with the cache, and single-pass becomes the only path for every caller.

Start at pkg/build/build.go:53 and trace compressionCache through Digest(), Size(), and its Store/Load operations. Measure cache hits and misses with representative image builds, then compare the two-pass and single-pass paths. Done means choosing between removing the cache and lazy descriptor machinery, or documenting and bounding the cache if it provides a measurable benefit.