monad-developers / monad-developers/ultrafuzz
stats counts operator-cancelled nodes as failed while status counts them as other
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 85
- Forks
- 18
- Avg merge
- 11h 10m
- Merged PRs (30d)
- 194
Description
Summary
ultrafuzz status and ultrafuzz stats disagree about the same run. A node that was interrupted by an operator-requested ultrafuzz cancel is counted as other by status and as failed by stats.
Asking "did anything fail?" gives a different answer depending on which command you ask.
Observed
One run, cancelled on a timer while two nodes were mid-flight:
$ ultrafuzz status --project . linux-run2
Progress: 3% (8 finished / 0 running / 212 pending / 0 failed / 2 other / 222 total)
$ ultrafuzz status --project . linux-run2 --json
{"percent": 3, "finished": 8, "in_progress": 0, "pending": 212,
"failed": 0, "skipped": 0, "remaining": 214, "total": 222}
$ ultrafuzz stats linux-run2 --project .
12 succeeded
2 failed <-- threat-model, setup-foundry
77 pending
status is right. Both nodes were interrupted by the cancel, not by any failure of their own:
error: "CLI aborted See https://smithers.sh/reference/errors"
warn child process interrupted process:/usr/local/bin/bun runId=ultrafuzz-linux-run2
Neither node had failed on its own terms; each was executing when cancellation arrived.
Why it is worth fixing
A cancelled campaign is a normal outcome -- a time-boxed run, a budget stop, an operator changing their mind. Reporting the nodes that happened to be executing at that moment as failed invites a hunt for a defect that does not exist. I went looking for two bugs before finding CLI aborted in the log and realising the cancel had caused both.
It also makes the two views hard to reconcile: status says nothing failed, stats names two failures, and nothing in either output explains the difference.
stats additionally lists 91 rows against a 222-node graph -- presumably only nodes with recorded attempts -- which is reasonable on its own, but combined with the conflicting failed count it makes the two commands look like they are describing different runs.
Expected
A node interrupted by cancellation should be reported distinctly from one that failed -- cancelled, aborted, or interrupted -- in stats as it already is in status. Failing that, the two commands should at least agree on the count.
Environment
Host — macOS 26.6.2 (Darwin 25.6.0) arm64; podman 6.0.2 client, applehv machine, 4 CPU / 8 GiB / 60 GB
Podman VM — podman server 6.0.2 linux/arm64; kernel 7.1.3-200.fc44.aarch64 (Fedora CoreOS 44); crun 1.28
Container — docker.io/library/node:24-bookworm (sha256:78b162211207872503ea9245188122b815150b9b4380e47a7c4a447332c01660), Debian GNU/Linux 12 (bookworm), aarch64, running as uid 1000, --memory 7g --cpus 4
Toolchain — node v24.20.0, pnpm 11.1.1, bun 1.4.0, forge 1.8.1, git 2.39.5, claude 2.1.258 (subscription, authMethod: claude.ai)
Ultrafuzz — main (8b985b90) plus #1083, built in-image
Target — morpho-org/metamorpho-v1.1 @ 89de526, default audit profile, claude-opus-5
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how ultrafuzz status and ultrafuzz stats classify node outcomes, using the reproduced cancellation commands and output as the behavioral baseline. Verify the handling of CLI aborted and interrupted child processes; done means cancelled nodes are represented distinctly or both commands report the same count, with coverage for an operator-cancelled run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100