monad-developers / monad-developers/ultrafuzz
final-report run-metadata authority derives task metrics during preparation, so every target fails on the Smithers task runtime
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 85
- Forks
- 18
- Avg merge
- 11h 10m
- Merged PRs (30d)
- 194
Description
Symptom
Every final-report node fails, on every smoke target, with:
Smithers task runtime is only available while a builder step is executing.
See https://smithers.sh/reference/errors
Dispatched run 33464828567 on ff274666: planned 3, launched 3, launch_failed 0, workflow_failed 3, workflow_nonterminal 0, terminal_reports_present 0, scoring_ready false. All three rows are identical, so this is deterministic, not a flake.
Mechanism
materializeFinalReportRunMetadataAuthority (packages/runtime/src/templates/smithers/workflows/workflow.tsx:2523) is invoked at :3419, at the end of a preparation routine — its immediate neighbours are restoreWorkspacePatchPreparation, prepareArtifactMirror and materializePromptArtifactAuthority, i.e. this runs in prepare:*, not in a builder step.
It awaits deriveAuthoritativeFinalReportWorkflowMetrics (:2429), which calls deriveCurrentTaskWorkflowMetrics() (packages/runtime/src/workflow-task-metrics.ts:267), which awaits readCurrentTaskWorkflowEvidence() (:178). That reads the current Smithers task's durable usage and timing evidence, which the engine only exposes while a builder step is executing — hence the throw.
So the authority materialization is correct in intent but runs one phase too early: it needs the task runtime, and preparation does not have one.
Introduced by
ff274666 (#1007, "fix(report): recover metrics and simplify developer output"), specifically its "recover authoritative run metrics" / "preserve relocatable workflow metrics" commits, which added workflowMetrics to the run-metadata projection:
+ const workflowMetrics = await deriveAuthoritativeFinalReportWorkflowMetrics(task);
+ const projection = deriveAuthoritativeFinalReportRunMetadata(task, workflowMetrics);
The immediately preceding dispatched run 33452521409, on 7ca243f5 (the commit just before #1007), reached final-report on all three targets and failed on different, report-shape errors — never on the task runtime. So the regression window is exactly 7ca243f5..ff274666.
Expected
The run-metadata authority must be materialized where the task runtime exists. Options, in what I take to be preference order — but this is a design call for the author of #1007:
- Derive the workflow metrics inside the builder step that produces
report.jsonand thread them into the authority, keeping preparation runtime-free. - Split materialization: keep the filesystem/projection work in preparation, and attach the metrics in the builder step before the comparison.
Note that simply catching the error and treating metrics as absent would defeat #1007's purpose — deriveCurrentTaskWorkflowMetrics already returns undefined legitimately, so swallowing the throw would silently publish reports with no run metrics and make #1006 regress invisibly.
Why this matters now
This is the last thing standing between the smoke lane and its first published observation since 2026-08-12. With #1013 and #1027 in, the lane now clears cohort reachability, completes preparation on all three targets, runs the full agent workflow, and reaches the final node — workflow_nonterminal is 0 and no target dies early any more. Every target now fails only here.
Related
- #1012 — the original four-cause outage RCA
- #1026 — the validator preflight budget and final-report prompt contradiction (both fixed and confirmed in run 33452521409)
- #1028 — the staleness monitor's own tracking issue, which will close itself once an observation publishes
Also worth a small separate fix
packages/runtime/src/final-report-markdown.ts:169 throws canonical final report Markdown does not satisfy the final-review report shape: ${directiveViolation}, but the message that reaches public-eval-diagnostics.json is exactly 78 bytes and ends at "shape" with no colon — the violation detail never lands in the artifact, so the gate knows exactly which directive failed and does not say. That cost a diagnostic cycle in run 33452521409.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at materializeFinalReportRunMetadataAuthority in packages/runtime/src/templates/smithers/workflows/workflow.tsx and trace its preparation-time call through deriveAuthoritativeFinalReportWorkflowMetrics and packages/runtime/src/workflow-task-metrics.ts. Compare this with the builder step that produces report.json; done means final-report completes without the task-runtime error while authoritative metrics remain available for comparison.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100