Run ultrafuzz-bench smoke lane across max-reasoning model matrix

Goal

Run the ultrafuzz-bench smoke lane for the models that are not yet on the published README charts, and publish their results alongside the existing ones. This is the cheap, bounded counterpart to the full-lane matrix in #138.

The smoke lane gives a comparable signal per model — small topology, one loop, no invariant/differential/dynamic work — so we can extend the README chart coverage without paying for a full benchmark per model.

Already covered — do not re-run

The published history (benchmarks/history.json, 99 observations) already contains these models, and they must not be re-run:

• gpt-5.6-luna (90 observations)
• kimi-k3 (6 observations)
• deepseek-v4-pro (3 observations)

Their existing observations and chart entries must be preserved untouched. This issue only adds new rows.

Dependency

Blocked on DeepSeek V4 Flash being merged to origin/main (in flight in #165). Do not start any run before that lands, because the runner registry and pricing/telemetry plumbing for the DeepSeek V4 Flash work is part of it and the benchmark should start from a main that includes it.

Note that #165 publishes its own three deepseek-v4-flash observations (one each for Foundry/Hardhat/Vyper). So once it is merged, deepseek-v4-flash also falls under "already covered": verify those three observations are present on main rather than re-running Flash. Only run Flash here if that verification fails.

#138 is the full-lane counterpart. This issue does not block on #138; the smoke matrix is intended to run first and cheaply.

Benchmark scope

Use the existing smoke lane from benchmarks/lanes.json and do not invent a new config. Its defining knobs are:

• Workflow: benchmarks/smoke-benchmark.yml, profile smoke-benchmark-v1
• strategy_loops: 1
• trials_per_variant: 1
• disable_invariant_tests: true
• disable_differential_tests: true
• disable_dynamic_strategies: true
• Selected strategies only (4): time-warp-sequences, external-dependency-boundaries, externalized-state-accounting, lifecycle-view-boundaries
• Lane concurrency: max_parallel_runs: 3, max_parallel_targets: 4

Benchmark target set (smoke_targets, all three so we keep Foundry/Hardhat/Vyper coverage):

• very-liquid-vaults-foundry
• venus-isolated-pools-hardhat
• stableswap-ng-vyper

The smoke lane currently pins a single runner profile (benchmark-smoke-gpt-5-6-luna-high). Each model below should run as an explicit runner model-profile override, which the manifest already validates, keeping the runner distinct from the judge.

Judge stays fixed at benchmark-judge-gpt-5-6-sol-xhigh (gpt-5.6-sol at xhigh) for every row, so results are comparable with what is already published.

Model matrix to run

Only the models missing from the charts, all at maximum reasoning available for that model/provider:

• gpt-5.5 at xhigh (CodexAgent)
• gpt-5.6-sol at xhigh (CodexAgent)
• gpt-5.6-terra at xhigh (CodexAgent)
• claude-fable-5 at max (ClaudeAgent)
• claude-opus-5 at max (ClaudeAgent) — not currently in the model registry on main, which carries claude-fable-5, claude-opus-4-8, and claude-sonnet-5. Adding that profile, or substituting the closest supported Opus slug and saying so, is a prerequisite for this row.

That is 5 models x 3 targets = 15 smoke runs.

Execution requirements

• Run on Modal.
• Use API-key billing, not subscription forwarding. Credentials come from ~/.bashrc: OPENAI_API_KEY for the GPT rows, ANTHROPIC_API_KEY for the Claude rows, DEEPSEEK_API_KEY if a DeepSeek row is needed, and MODAL_TOKEN_ID / MODAL_TOKEN_SECRET for Modal. We always want metered API billing for these runs so cost is attributable per model. Never print, log, or commit any credential value.
• Use the smoke lane's own bounded concurrency (max_parallel_runs: 3, max_parallel_targets: 4). Do not raise it to full-lane concurrency; the bound is part of what makes this lane cheap and reproducible.
• Start from origin/main after DeepSeek V4 Flash has landed.
• Preserve every smoke lane knob above. The lane guardrails are enforced in packages/evals/src/benchmark-manifest.ts, so a deviation should be treated as a defect unless explicitly justified and documented.
• Vary only the runner model profile between rows. Keep targets, strategies, loops, and judge identical.

Acceptance criteria

• Each of the 5 models has a completed Modal smoke run across all three smoke targets, or a clearly classified provider/auth/infrastructure failure.
• Results are published into benchmarks/history.json and the eval-history charts in the same established style as the existing entries, and the README charts render the new models.
• The pre-existing gpt-5.6-luna, kimi-k3, and deepseek-v4-pro observations are unchanged, and the three deepseek-v4-flash observations from #165 are verified present.
• Results include Modal run IDs or URLs, per-model and per-target status, aggregate findings, and any failures/retries.
• The final report states the exact git commit, lane (smoke), workflow profile (smoke-benchmark-v1), loop count, selected strategy set, model slugs, reasoning settings, judge profile, auth mode (API-key billing), and concurrency settings used.
• The report confirms the lane guardrails actually held for every run: strategy_loops: 1, and invariant, differential, and dynamic strategies all disabled.
• Per-model cost and wall-clock time are recorded, so the smoke lane can be used to forecast the cost of the full matrix in #138.
• Any benchmark-runner fixes needed to support the matrix are captured in a separate PR or documented as blockers.

Start by verifying #165 is merged to origin/main and checking benchmarks/history.json for the existing observations. Read benchmarks/lanes.json, benchmarks/smoke-benchmark.yml, and packages/evals/src/benchmark-manifest.ts before running the five model profiles across the three listed targets. Done means the smoke results and charts are published with the required run metadata, costs, guardrail confirmation, and unchanged prior observations.