google / google/capsem

Preserve host-builder cache across the canonical just test gate

Open
#166 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
72
Forks
13
Avg merge
1d 2h
Merged PRs (30d)
5

Description

## Problem

The canonical local `just test` builds `capsem-host-builder` in its Linux parity stages, then `test-assets` calls `scripts/ensure-docker-space.sh 16`. When daemon free space is below the threshold, that script runs `docker builder prune -af`.

In the July 18 physical-Mac Ironbank run, this deleted the layers for the identical host-builder image. Stage 7 immediately called `just cross-compile arm64`, rebuilt all 16 Dockerfile layers from scratch, and spent 255 seconds recreating the image before compiling either package. The gate therefore destroys a cache it knows a later required stage will consume.

The same audit found that `ensure-docker-space.sh` probes daemon space with `docker run --rm debian:bookworm-slim` without an explicit native `--platform`; `check_container_clock()` similarly runs an unqualified `alpine` tag. A cached foreign-architecture tag can therefore silently invoke QEMU/Rosetta in local or CI helper paths.

## Required outcome

- The local Ironbank gate may reclaim enough Docker space for both asset lanes without deleting the exact host-builder cache required by later stages.
- Cache cleanup must be ownership-aware and fail closed if the required disk cushion cannot be achieved safely.
- Native helper containers must normalize x86_64/amd64 and arm64/aarch64 to an explicit Linux platform and reject unknown host architectures.
- Preserve the release rule: `just test` still executes every rail; this issue only removes self-inflicted rebuilds and foreign-architecture helper execution.

## Acceptance criteria

- A contract test proves `test-assets` space recovery preserves the host-builder cache needed by Stage 7.
- A local full-gate trace proves Stage 7 reuses the image built earlier in the same `just test` invocation.
- Adversarial tests cover low-space cleanup, insufficient-space failure, both host-architecture alias pairs, unknown architecture, and a cached foreign image tag.
- `ensure-docker-space.sh` and container-clock diagnostics explicitly select the native platform.
- The release-process and dev-testing skills document the cache-ownership and explicit-platform contracts.

Contributor guide

Open the contributing guide

Research direction

Start with scripts/ensure-docker-space.sh, the container-clock diagnostic, and the canonical just test flow, including test-assets and just cross-compile arm64. Trace cache cleanup and helper-container platform selection, then use the stated adversarial tests and a full-gate trace to verify cache reuse, safe failure, native architecture handling, and the documented release-process and dev-testing contracts.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, rust, shell
Domain
build-system, devops, documentation, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.