monad-developers / monad-developers/ultrafuzz
Terminal workflow stays running in status after unrecorded attempt supersession
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 85
- Forks
- 18
- Avg merge
- 11h 10m
- Merged PRs (30d)
- 194
Description
A recovered workflow can finish every remaining task and publish a verified final report while ultrafuzz status --json still returns ok: false, status: running, workflow_status: finished, zero remaining tasks, and ETA zero.
The observed diagnostic was:
WORKFLOW_ATTEMPT_INSPECT_FAILED
Smithers attempt ["node:synthetic-worker",0,1] supersedes terminal event <sequence> before durable attempt recording
RUN_WORKFLOW_STATUS_DIVERGED
Ultrafuzz run is running while the workflow runner reports terminal finished
The node name and event sequence above are placeholders. The workflow had undergone recovery that reused attempt numbers; the final producer and verifier were complete. This is a historical bookkeeping failure, not unfinished report generation. The report and native bundle were independently readable and verified.
On current main, terminalWorkflowAttempts in packages/runtime/src/workflow-sync.ts rejects a superseded terminal occurrence unless it was already recorded or an explicit authority callback admits it. In the observed recovery history, that exception prevents terminal run-state reconciliation. The exact missing authority needs an isolated reproducer; this report does not propose accepting arbitrary supersession.
Suggested regression fixture:
- Use a model-free workflow that finishes an attempt.
- Retain that event history, then reopen the same node/iteration/attempt identity through a supported recovery boundary before its old terminal occurrence is projected.
- Finish the replacement and final report, then synchronize status and stats twice.
Expected: retain each authenticated historical occurrence under a distinct durable identity and report terminal completion consistently. If part of the history remains unrecoverable, expose a bounded history diagnostic without presenting the already-finished runner as actively executing. Observation must not reset tasks or rewrite historical results.
Related: #951, #1036, #1077. This issue tracks the terminal-reconciliation impact and the unrecorded-supersession diagnostic, rather than the different immutable-ledger collision in #1036.
Sanitized: no target names, findings, run identifiers, host paths, credentials, or raw campaign artifacts are included.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in packages/runtime/src/workflow-sync.ts at terminalWorkflowAttempts and trace the status and stats synchronization path. Build the suggested model-free recovery history with a reused attempt identity, then synchronize twice. Done means authenticated historical occurrences retain distinct durable identities, terminal completion is reported consistently, and unrecoverable history produces a bounded diagnostic without resetting tasks or rewriting results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100