[SPIKE] Investigate parallel push + sorted re-push to speed up publish stage

Background

To ensure Docker Hub displays the correct/most-recent tag by default, we push image tags in a specific sorted order (see #484) rather than in parallel. This guarantees the last layer pushed corresponds to the newest version, keeping Docker Hub's ordering sane for users browsing tags.

The tradeoff is that the sorted/ordered push is effectively serialized, which slows down the overall publish stage compared to pushing all tags in parallel.

Idea

Investigate whether we can get the speed benefits of parallel pushing and keep correct sort order by doing a two-phase push:

1. Parallel push phase: Push all image tags to Docker Hub in parallel (order not guaranteed), to get the bulk of the layer data uploaded as fast as possible.
2. Sorted re-push phase: After the parallel phase completes, re-push the tags again in the correct sorted order. Since the layers already exist in the registry from phase 1, this second push should be nearly instant (Docker Hub/registry layer dedup means only manifest updates are needed, no blob re-upload) while still producing the final correct tag ordering.

If this works as expected, we get the best of both worlds: the fast wall-clock time of parallel pushing, with the final state matching what a fully serialized/sorted push would produce.

Goals of this spike

• Confirm Docker Hub / registry behavior: does re-pushing a tag whose layers already exist in the registry actually skip layer upload and only update the manifest (i.e. is the second push fast)?
• Confirm that the final tag order on Docker Hub reflects the order of the second (sorted) push, not the first (parallel) push.
• Prototype this in bakery (or wherever the current publish/push logic lives) and measure end-to-end publish stage time before/after.
• Identify any edge cases (e.g. partial failures in phase 1, manifest lists/multi-arch images, rate limiting from double-pushing).
• Write up findings and a recommendation (adopt, adopt with caveats, or not worth it) for follow-up implementation work.

Related

• #484 — Push image versions in reverse order (establishes why push order matters for Docker Hub)

Start by reading #484 and locating the current publish/push logic in bakery. Measure the existing sorted push, then prototype the parallel phase followed by a sorted re-push and compare publish times. Confirm Docker Hub tag ordering, layer deduplication, and the listed failure, multi-architecture, and rate-limit cases; document findings and a recommendation.