A TEST_CASE that MESSAGEs SKIP and returns is counted as a PASS with zero assertions: 228 sites, and the SACRED DeepSeek-V2 gate is one of them

Row: -

Listed under ## Owed O4 of .agents/specs/glm4-moe-lite-gate-2839.md.

What

A TEST_CASE that emits MESSAGE("SKIP ...") and returns is counted by
doctest as a passed test case with zero assertions. The run prints
Status: SUCCESS!. Nothing distinguishes it from a case that ran and proved
something, and no gate reads the assertion count.

Measured on 3f12c617d, a host with no large checkpoints:

$ ./build/tests/test_deepseek_v2_paged_engine
[doctest] test cases: 1 | 1 passed | 0 failed | 0 skipped
[doctest] assertions: 0 | 0 passed | 0 failed
[doctest] Status: SUCCESS!

That is the SACRED DeepSeek-V2 correctness gate reporting SUCCESS having
measured nothing at all.

How wide

MESSAGE(...SKIP...); return; occurs at 228 sites in 72 files under
tests/. It is not confined to checkpoint-gated model gates; the largest single
concentration is tests/vt/test_tenstorrent_backend.cpp (54 sites), then
tests/vt/test_ops_attention_cross.cpp (15) and
tests/vllm/models/test_ltx2_device.cpp (9).

Not every site is equally wrong. Some sit after real assertions in the same case,
so the case still reports a non-zero count. The defect is the shape: an early
return cannot be told from a pass, so whether any given site is a lie depends
on what ran before it, and nothing checks.

Why it matters

This is the mechanism that let
#2839,
#2840 and
#2841 stand: a gate that never
runs is never noticed, so a gate that runs and asserts nothing is never noticed
either. #2839 repaired one instance --
test_glm4_moe_lite_paged_engine now uses doctest::skip, so the case reports
1 skipped instead of 1 passed | 0 assertions -- and finding the underlying
defect took reading the goldens by hand rather than reading a gate result.

What would close it

Either of these, and the choice is a product decision this issue does not make:

1. Convert the checkpoint/environment predicates to doctest::skip(...), so an
   absent resource reports as skipped. test_kimi_linear_fold_gate already
   does this and is the in-tree precedent.
2. A checker that refuses a test binary reporting assertions: 0 for a case that
   was neither declared skipped nor filtered out. This catches the shape rather
   than each site, but it needs a decision about which binaries are legitimately
   assertion-free.

Repairing 228 sites is a campaign, not an in-flow fix, which is why this is filed
with an owner rather than fixed in the flow that found it.

Related: #2839, #2840, #2841, #2834, #2605.

Start by reviewing the 228 MESSAGE("SKIP ...") sites under tests/, especially tests/vt/test_tenstorrent_backend.cpp, tests/vt/test_ops_attention_cross.cpp, and tests/vllm/models/test_ltx2_device.cpp. Read the precedent in test_kimi_linear_fold_gate and the related work in #2839. Done means the chosen policy distinguishes skipped or assertion-free cases from successful tests and covers the affected sites or binaries.