Rules.md 4.3.5: a missing "not" inverted subset-run validation — subset mode is enforced backwards

Summary

Rules.md §4.3.5 (checkpointSubsetRunValidation) is missing a single word — "not" — and that omission inverted the rule's meaning everywhere downstream: in Table 2, in the tool's run-time gates, and in the submission validator. As published:

The submission validator must flag an error if the subset argument is given but the total number of accelerators is not exactly 8, or the model is "8B".

The intended text is:

…or the model is not "8B".

Subset mode exists for exactly one purpose: letting a storage architecture that centrally manages client-local NVMe — whose checkpoint bandwidth is trivially linear in node count — demonstrate scale-out from a single 8-GPU node running the 8B workload. The larger models (70B / 405B / 1T) exist to measure architectures where checkpoint data must cross a network to shared storage; no subset form was ever defined for them. With the "not" present, subset + any larger model is an illegal combination.

How the missing word would have shown the contradiction

Table 2's last row, "Subset: 8-Process Size", publishes computed sizes for all four models (105 GB / 114 GB / 94 GB / 161 GB). Had 4.3.5 carried the "not", that row would have contradicted the rule on its face — a subset size for 70B/405B/1T describes a run the validator must reject, so those cells could never have held valid values. The row went unchallenged precisely because the typo'd rule text reads as endorsing 8-accelerator subset runs of the large models: it errors only on subset ∧ accelerators ≠ 8 and subset ∧ model == 8B, which makes the large-model subset run the only well-formed subset. The table and the rule were consistent with each other — and both wrong.

Where the inversion landed in code

• submission_checker/checks/checkpointing_checks.py — subset_run_validation implements the typo faithfully: it errors "subset run cannot use 8B model" (backwards — that is the only legitimate subset) and passes subset runs of 70B/405B/1T with 8 accelerators.
• submission_checker/checks/checkpointing_checks.py — closed_mpi_processes (§4.6.1) carves out subset mode: "requires exactly 8 processes for any model", defeating the respective-count requirement (8/64/512/1024).
• rules/run_checkers/checkpointing.py — check_num_processes (the #792 fail-fast gate) accepts 8 processes as a valid CLOSED form for every model, so mlpstorage checkpointing run closed --model llama3-405b --num-processes 8 launches without complaint.
• The explicit subset CLI parameter that 4.3.5's first sentence requires was never implemented — checkpoint_subset help text sits orphaned in cli/common_args.py; subset-ness is only ever auto-inferred (benchmarks/dlio.py:add_checkpoint_params, any num_processes < ClosedGPUs).

Impact on v3.0

Ten published CLOSED rows are subset runs of large models — 70B ×4, 405B ×3, 1T ×3, across three organizations — which the validator passes silently today. Disposition of those rows is a review-committee matter, tracked separately.

Fix (in progress on reportgen-column-parity)

1. Rules.md: insert the missing "not"; mark Table 2's large-model subset entries Invalid.
2. CLI: implement --checkpoint-subset (valid only with the 8B model; a pure claim marker — 8B subset is execution-identical to the full 8B run).
3. Run checker: subset ∧ model ≠ 8B and subset ∧ processes ≠ 8 → INVALID (launch aborts); 8-process large-model runs are no longer a CLOSED form (70B@8 is a TP×PP multiple → OPEN-eligible; 405B/1T@8 → INVALID).
4. Validator: fix the inversion in 4.3.5; remove the 4.6.1 subset carve-out. Downscaled OPEN runs remain legal per §4.6.4 (they use the same partial-checkpoint mechanics but are governed by the multiples rule, not 4.3.5).

Start with Rules.md §4.3.5 and the named checkpointing checks in submission_checker/checks/checkpointing_checks.py, then inspect rules/run_checkers/checkpointing.py and cli/common_args.py. Verify the existing subset inference in benchmarks/dlio.py. Done means the rule, CLI, run checker, validator, and Table 2 treatment agree on valid 8B subset runs and reject the specified invalid combinations.