Fix loop: `initial_code.py` has no write-once contract, so the Stage-0 baseline gets overwritten in place and "initial == final" is unmeasurable

Summary

The LIBERO fix loop asks the subagent to write a Stage-0 baseline program, initial_code.py, and sweep it over the development seeds. Nothing after that protects the file: no hash, no copy, no read-only bit, and no instruction to leave it alone. In my libero_goal_swap campaign (10 tasks) the subagent overwrote initial_code.py in place on 4 of 10 tasks after the initial sweep had already run, so on those tasks initial_code.py is byte-identical to fix_code.py and the pre-debug program only survives inside the per-trial code.py copies.

Two consequences:

1. "Initial equals final" cannot be interpreted. Six of ten tasks ended byte-identical. Checked against the per-trial evidence, those six decompose as 4 overwritten, 1 genuinely unchanged, 1 unverifiable. From the two files alone you cannot tell "the loop changed nothing" from "the loop destroyed its baseline".
2. Anything that replays initial_code.py as the un-debugged baseline re-runs the debugged program. I had a control arm doing exactly that. On put_the_bowl_on_the_plate it read 15/15 vs 15/15 against the Stage-2 result, which invites "debugging bought nothing", when the dev seeds had actually run a different, 11,636-byte program.

I am filing this as reporting hygiene, not data loss: nothing corrupted is committed (outputs/ is gitignored, .gitignore:34). The latent defect is in the prompt on main.

Where it comes from

• aspire/sim/.claude/libero/fix-loop/subagent-prompt.md:84 — "Write $TASK_DIR/initial_code.py using only allowed APIs." The subject of the experiment writes its own baseline into its own writable working directory.
• :88-98 — the initial sweep replays it on seeds 51–65 and logs to $TASK_DIR/initial_seed_<seed>.log.
• Stage 1, Step 3 — fixes are tested from a reused scratch path (/tmp/fix_attempt.py); the prompt says to "improve ONE task-level program". An agent that edits initial_code.py in place is following the letter of the prompt.
• :180-184 — Step 4 writes $TASK_DIR/fix_code.py. If the agent iterated in place, this is a copy of initial_code.py, and the baseline is gone.

There is no buggy line to point at. The absent contract is the finding.

Evidence

The per-trial evidence exists because scripts/libero/replay_trial.py:550 writes the executed program into every trial directory as code.py (with # Code block N headers; one header line plus the stripped trailing newline makes it 14 bytes longer than the source file). Each initial_seed_<seed>.log prints the trial directory on an Output: line (replay_trial.py:594), so a sweep's trial dirs can be bound to the sweep without guessing.

In all four overwrite cases the file's mtime is later than the entire sweep it is supposed to describe, and its contents differ from what every trial in that sweep ran. The other four tasks (initial ≠ fix) all match their sweep byte for byte, which is what validates the comparison.

Two traps worth recording for anyone who re-checks this:

• Comparing initial_code.py to fix_code.py by mtime is wrong in both directions. put_the_wine_bottle_on_the_rack has the two files 77 minutes apart yet is overwritten; put_the_wine_bottle_on_top_of_the_cabinet has them 4 minutes apart yet is clean. Only the contents of the sweep's own code.py decide it.
• Trial directories are keyed on (seed, reward) and every sweep writes into the same --output-dir. A later replay that reproduces a reward silently replaces the earlier code.py. On push_the_plate seeds 51 and 61 were re-run after the overwrite and now hold the new program, so a naive check calls that task "mixed". put_the_bowl_on_the_stove is unverifiable for the same reason: its initial_code.py and fix_code.py are byte-identical with the same mtime, so a surviving match proves nothing.

Repro

cd aspire/sim
T=outputs/libero_fix_loop/libero_goal_swap/put_the_bowl_on_the_plate
for f in $T/initial_seed_*.log; do
  od=$(grep -m1 -E '^\s*Output: ' "$f" | sed 's/^\s*Output: //')
  sha256sum "$od/code.py"
done | sort | uniq -c          # 15 × one sha, 11,650 B
sha256sum $T/initial_code.py   # a different sha, 12,635 B

Suggested fix

The repo already has the pattern: scripts/libero/run_fix_loop_validation.py:50 writes "code_sha256": sha256_file(fix_code) into a manifest. Applying it to Stage 0:

1. After the smoke test and before the sweep, record sha256(initial_code.py) in a task manifest and chmod 444 the file; state in subagent-prompt.md that it is frozen after Stage 0.
2. Have Stage 1 write each tested program once, as $TASK_DIR/attempts/attempt_<NNN>.py, instead of a reused /tmp/fix_attempt.py, and make fix_code.py a copy of the winning attempt rather than an in-place edit. The attempts/ directory already exists for BLOCKED.md.
3. Optionally have replay_trial.py record the sha256 of --args.replay-code in summary.txt, so a trial can be bound to its program without normalising code.py.

A related site with the same shape: scripts/libero/evosearch_eval.py:182 writes code.py raw, without the # Code block N headers replay_trial.py emits, so the two harnesses' code.py files are not directly comparable.

Happy to send a PR for (1) and (2) if that is the direction you want.

cc @LRY89757

Read aspire/sim/.claude/libero/fix-loop/subagent-prompt.md around lines 84–98 and 180–184, then inspect scripts/libero/run_fix_loop_validation.py:50 and scripts/libero/replay_trial.py:550,594. Reproduce the overwrite with the provided command. Done means the Stage-0 baseline is hash-recorded and frozen, each Stage-1 attempt is retained separately, and the resulting files and manifests make initial-versus-final comparisons unambiguous.