Fix CI runner environment isolation and container smoke database setup

Goal

Restore the two failing CI harness boundaries observed in main run 34060866329: synthetic local runner tests must execute against their local machine, and the non-root release-image smoke must use a writable, migrated scratch SQLite database before checking the real /unified/ page. This unblocks migration bootstrap #318.

Normative references

• AGENTS.md, _docs/PROCESS.md, .claude/agents/software-engineer.md, .claude/agents/tester.md.
• _docs/ci/change-selective-ci.md: exact component environment, independent evidence, report and graph-driven checks.
• _docs/specs/07-security-privacy-operations.md, _docs/specs/08-aws-development-terraform.md, _docs/architecture/database-portability.md.

Verified source and diagnosis

On-call reproduced three failures under VERIFICATION_CONTAINER_ARCHITECTURE=aarch64 VERIFICATION_CONTAINER_RUNNER_IMAGE=ubuntu24-arm64:

• tests_ci/test_runner.py::test_runner_records_the_selected_tester_role.
• tests_ci/test_runner_timeout.py::test_run_plan_bounds_a_hanging_component_and_still_emits_the_report.
• tests_ci/test_runner_timeout.py::test_run_plan_green_run_is_unaffected_by_the_timeout_plumbing.

Both modules' plan_for helpers call ci.verification.build_plan, inheriting the real workflow's remote container target while their run_plan invocation executes components on the local host. These are local synthetic execution scenarios, so they must explicitly isolate those two declarations in test fixtures before planning. Production declarations and ci.runner environment verification are correct and remain unchanged.

The Smoke-test liveness without publishing step in .github/workflows/ci.yml starts dtc-website:$RELEASE_SHA web under website.settings.local, with UID/GID 10001. website/settings/local.py defaults SQLite to /app/.tmp/local.sqlite3; the workflow currently neither mounts writable scratch storage nor applies migrations. /health/live works; the actual /unified/ page fails 500. ci/container_check.py already demonstrates the correct runtime setup: process-scoped repository .tmp directory, writable to the image's non-root user, mounted at /app/.tmp, and uv run --no-sync python manage.py migrate --noinput run in the same image before starting web with the same mount.

Scope and ownership

One gpt-5.6-luna engineer at max reasoning, isolated worktree on approved current main. Engineer is not alone; preserve other agents' edits.

Own:

• tests_ci/test_runner.py and tests_ci/test_runner_timeout.py: narrowly scoped synthetic local-execution fixture isolation.
• .github/workflows/ci.yml: the existing normal container smoke step only, plus minimal same-step evidence assertions required by the real migration setup.
• ci/tests/test_workflows.py and core/tests/test_deployment_workflow.py: extend existing container/workflow contracts with the scratch mount, same-image migration-before-web and real page checks.
• A small local helper/test fixture is permitted only if these exact owners consume it; prefer existing module-local fixtures to global test environment mutation.

Read without changing ci/container_check.py, ci/runner.py, ci/verification.py, tests_ci/test_verification.py, website/settings/local.py, Dockerfile, entrypoint.sh and relevant Make verification targets. An unavoidable production-code change returns to PM with concrete evidence; do not silently broaden scope.

Steps

1. Record base/head/worktree status and rerun the three named tests with inherited target declarations. Retain the failure diagnosis without secrets. A failure attributable to a new unrelated baseline issue is reported, not rewritten into this scope.
2. Isolate VERIFICATION_CONTAINER_ARCHITECTURE and VERIFICATION_CONTAINER_RUNNER_IMAGE in the local synthetic runner fixtures before their plans are built. Use pytest monkeypatch so environment restoration is automatic. Do not alter real ImageOS/ImageVersion or mock away environment comparisons. Tests explicitly exercising declared target mismatch in tests_ci/test_verification.py retain their deliberate environment values.
3. Run the runner modules with both remote declarations present externally and with both absent. The role test must still produce tester-origin envelopes; timeout must still terminate the hung component, preserve partial output, run remaining components and produce a failure report; green run must produce success. Add only a meaningful regression assertion/parameterization if needed to ensure isolation cannot accidentally change production environment validation.
4. In the workflow smoke step create a fresh run-scoped scratch directory under project .tmp, writable by UID/GID 10001. Follow ci/container_check.py's writable scratch convention; never chmod the application root or change container user. Construct one bind mount to /app/.tmp, use it for both a one-shot migration container and the web container, and use the exact same dtc-website:$RELEASE_SHA image/local settings for both. Execute uv run --no-sync python manage.py migrate --noinput via explicit entrypoint before web starts. Migration failure must fail the step before declaring health success.
5. Retain existing VERSION/SOURCE_SHA/IMAGE_DIGEST injection and exact /health/live JSON assertion, real /unified/ HTTP success check, runtime UID/GID assertions, static-manifest negative checks, cleanup/log trap and single-image preservation. Record scratch migration proof in actual container assertions if required by the maintained evidence schema. Do not fake HTTP responses or change /unified/ to a DB-free endpoint.
6. Extend workflow contract tests to check migration occurs before the web command, both use the same image and scratch mount, the mount is writable to the existing non-root UID, and real liveness/page checks remain. Keep test_serving_entrypoint_never_runs_migrations: migrations belong to this one-shot smoke command, never application web startup.
7. Execute focused tests, then the actual generated verification plan for the candidate. Independently exercise the changed shell step against the built release image with local synthetic SQLite and record real health identity plus /unified/ success. Existing make verification-container is relevant same-image evidence, but by itself does not prove the edited workflow block ran; the tester must exercise that block or an exact extracted equivalent, retaining commands/output. Negative test: failed migration prevents web success; unavailable database/page error or wrong identity remains a failing gate. No production inputs.
8. Freeze the uncommitted engineer handoff with exact base/head, graph/plan digests, commands/counts and four buckets. Separate tester verifies; PM accepts; engineer commits; orchestrator merges/pushes; on-call alone observes the resulting main CI and deployment. #318 waits for this gate to be green.

Explicit non-goals

No application, user-model, public-route or database schema change; no production access; no package migration/bootstrap; no relaxed environment fingerprint, cross-architecture/family allowlist, report validation or static/liveness gate; no CI redesign or global deletion of target environment variables; no migration inside entrypoint.sh; no running the web image as root.

Acceptance criteria

• Three named local runner tests pass with externally inherited ARM target declarations and with them absent, while retaining tester provenance, timeout/failure/remaining-component and success assertions.
• Tests in tests_ci/test_verification.py still prove correct declared ARM target acceptance and reject wrong architecture, wrong runner family and unreviewed declarations; test_runner_rejects_a_different_actual_runner_before_component_execution remains effective.
• Normal workflow creates isolated writable scratch storage under .tmp and migrates it in the exact release image before web startup; web uses the same storage as UID/GID 10001.
• A fresh real image smoke returns the exact release identity from /health/live and HTTP 200 from /unified/, with applied migration graph. A migration failure or wrong health/page result fails rather than reporting success.
• Existing static-manifest invalid/missing cases, image/runtime non-root checks, serving-entrypoint-no-migrations contract, cleanup and single-image preservation remain intact.
• Workflow/test contract checks and graph-selected verification pass; every component has a valid disposition and evidence; no required skip or pending product screenshot remains in tester-final.
• Independent tester passes and PM accepts before engineer commit; on-call reports restored required main CI/development checks. #318 is unblocked only after that green verdict.

Exact focused verification

VERIFICATION_CONTAINER_ARCHITECTURE=aarch64 VERIFICATION_CONTAINER_RUNNER_IMAGE=ubuntu24-arm64 uv run --frozen pytest tests_ci/test_runner.py tests_ci/test_runner_timeout.py tests_ci/test_verification.py -q
env -u VERIFICATION_CONTAINER_ARCHITECTURE -u VERIFICATION_CONTAINER_RUNNER_IMAGE uv run --frozen pytest tests_ci/test_runner.py tests_ci/test_runner_timeout.py -q
uv run --frozen pytest ci/tests/test_workflows.py -q
uv run python manage.py test core.tests.test_deployment_workflow --settings=website.settings.test
make test-ci
make verification-plan VERIFY_ISSUE=319
make verification-run VERIFY_ISSUE=319
make verification-evidence-check
make verification-report-check

Expected: zero unexpected failures/skips, actual negative rejection assertions remain asserted; runner timeout test deliberately produces a failing fixture report but its pytest test passes by verifying that behavior. Fill in current test counts from output, do not assume old 612 count after adding coverage. Graph impact from workflow/test infrastructure may select broad checks; honor it and reuse only exact validated evidence. Required tester commands use VERIFY_CONSUMER=tester VERIFY_PRODUCER_ROLE=tester VERIFY_PHASE=tester and the same frozen base/head.

Browser/visual: no product render change, so screenshots are not_applicable when the graph agrees. Real container /unified/ and liveness checks are required and cannot be waived as 'docs only'. Any selected Playwright tier remains required by the graph.

Rollback: revert this focused harness commit through normal site process; scratch DB is disposable and never copied into the image or production. Stop for repeated failed verification after genuine fix attempts, an inability to execute the real container test, or evidence that a production-code change is required.

Dependencies

No missing product dependency. Coordinate a clean current main worktree with the plan-document merge. Blocks #318 (DTC D0.1a bootstrap); #317 documentation and AISL #1563 are independent.

Start with the three named runner tests in tests_ci/test_runner.py and tests_ci/test_runner_timeout.py, then inspect the smoke step in .github/workflows/ci.yml and its contracts in ci/tests/test_workflows.py and core/tests/test_deployment_workflow.py. Run the listed focused pytest commands first. Done means isolated local runner fixtures pass, the release image migrates the shared writable scratch mount before web startup, and real /health/live and /unified/ checks succeed.