workers benchmark: unvalidated --iterations yields NaN metrics with passed:true; concurrent ignores --iterations

Summary

workers benchmark on 2.1.2 is, unusually for this CLI, mostly correct — --type validates and rejects, --iterations genuinely drives the workload, --json matches the text, and --type all produces a coherent summary. Three defects remain, one of them CI-relevant.

(It does not exist on @alpha at all — see #208 §3; not repeated here.)

---

1. --iterations is unvalidated, and a zero-work run reports passed: true

-i	Count	Avg	Throughput	exit
nonsense	NaN	NaNms	NaN ops/s	0 ✅
0	0	NaNms	NaN ops/s	0 ✅
-5	-5	NaNms	-Infinity ops/s	0 ✅

$ agentic-flow workers benchmark --type trigger-detection -i nonsense

✅ Trigger Detection
   Operation: detect
   Count: NaN
   Avg: NaNms | p95: 0.000ms
   Throughput: NaN ops/s

The --json for that same call:

{
  "name": "Trigger Detection",
  "count": null,
  "totalTimeMs": 0,
  "avgTimeMs": null,
  "p50Ms": 0, "p95Ms": 0, "p99Ms": 0,
  "throughput": null,
  "passed": true,
  "target": 5
}

"passed": true is the real problem. A benchmark that executed zero operations is reported as having met its 5 ms target, with a green ✅ and exit 0. Anything gating CI on passed (or on the exit code) goes green having measured nothing — a typo in an -i value is indistinguishable from a successful run.

Suggested fix: reject a non-finite or <= 1 iteration count with a clear error and exit 1; failing that, passed must be false when count is not a positive finite number.

Minor, same output: p95Ms: 0 renders as a real number beside a null average — the percentiles are computed off an empty sample without being marked absent.

This is the same unvalidated-numeric-option family as --reward nonsense → Reward: NaN (#205) and --score 5 → 500% on a documented 0–1 range (#204).

---

2. concurrent silently ignores --iterations

Every other type honours the flag; concurrent is pinned at 20 regardless.

type	default count	with -i 7
registry	1500	21	honoured
agent-selection	1000	7	honoured
cache	100	7	honoured
memory-keys	5000	35	honoured
concurrent	20	20	ignored

No warning; the value is accepted and discarded.

---

3. --iterations is documented as a count but is actually a base multiplier

-h says Number of iterations (default: "1000"). Actual counts for -i 7 are 21 (registry, ×3) and 35 (memory-keys, ×5). The per-type multipliers may well be deliberate, but the option's meaning should say so, otherwise the reported Count contradicts the value the user passed.

---

What works (for the record)

• --type nonsense-benchmark → Unknown benchmark type: nonsense-benchmark, exit 1. Correct rejection.

• --iterations produces real measurements, not constants — the warmup curve is visible:

  -i	Count	Avg	Throughput	Memory Δ
  1	1	0.172ms	5,807 ops/s	0.14MB
  10	10	0.064ms	15,522 ops/s	-0.84MB
  1000	1,000	0.001ms	1,491,375 ops/s	0.04MB
  100000	100,000	0.000ms	5,928,211 ops/s	9.19MB

• All six types run; --json is valid and consistent with the text renderer (no [object Object], no field-name mismatch — contrast #198, #201, #202, #208).

Environment

agentic-flow 2.1.2 (global), Node 24, macOS 15.6, run in an empty scratch directory.

Start at the workers benchmark command entry point and trace --iterations through each benchmark type, especially concurrent and the output/pass calculation. Reproduce the shown nonsense, zero, negative, and -i 7 cases in 2.1.2; done means invalid counts fail, concurrent honors the option, and the help text matches the observed multiplier behavior.