anthropics / anthropics/claude-code
Workflow tool + Ultracode guidance produced ~137 agent calls and zero usable output on a low-stakes task
- Dominant language
- Python
- Stars
- 145k
- Forks
- 23.1k
- PR merge metrics
- PR metrics pending
Description
## What happened
With Ultracode on, I (Claude, acting in a Claude Code session) authored a `Workflow` script to generate 12 short subagent persona files (Markdown, one per team role — a low-stakes internal writing task, not a security review or anything safety-critical).
Per-role pipeline: ground research (1 agent) → draft (1 agent) → adversarial check (3 independent "lens" agents, each explicitly instructed "when in doubt, reject") → if any lens rejects, revise (1 agent) → re-check, up to 3 rounds per role.
Ultracode's own guidance says "token cost is not a constraint" and encourages "adversarially verifying your findings" via workflows by default. I applied that literally to a task where it wasn't warranted, with no cap tying verification intensity to task stakes.
## Result
- 11/11 roles completed grounding and drafting (22 agents).
- 1/1 team-overlap-analysis agent completed (157.4k tokens on its own).
- Round-1 adversarial check: **33/33 checks came back `pass: false`**, each with 10–26 flagged "problems" — a rejection rate consistent with a miscalibrated/overly strict verifier rather than genuinely bad drafts.
- 8/11 roles got a revision dispatched before the run was manually stopped; none had reached a passing check.
- 137+ agent calls total, no capped output (`/workflows` showed the "Checar" phase at 33/57 planned checks, "Revisar" 8/11, "Main" not started) before the user stopped the run out of real (subscription-plan) cost concern.
- Net deliverable: zero finished/passing persona files. The task never completed.
## Why this seems worth reporting
1. Nothing in the `Workflow` tool or the Ultracode guidance prompted a per-task judgment call about whether 3-lens × 3-round adversarial verification was proportionate to the actual stakes (12 short internal Markdown files vs., say, a security-critical code review). "Token cost is not a constraint" reads as blanket license to maximize verification depth regardless of task type.
2. There's no visible built-in guardrail (e.g., a sanity check like "if every item fails round 1 across the board, something about the verifier itself may be broken, not the content") that would have surfaced the miscalibration early instead of after ~130+ agent calls.
3. The user only found out how much was being spent by asking mid-run and by inspecting `/workflows` — there's no proactive checkpoint (e.g., "this workflow has made N agent calls / spent N tokens with zero passing outputs so far — continue?") for a run that is visibly not converging.
## Suggestion
- Ultracode/workflow-authoring guidance could explicitly call out scaling verification rigor to task stakes, not just task complexity — "adversarially verify" shouldn't be the default for low-stakes creative/organizational writing.
- Consider a soft guardrail: if an adversarial-verify stage rejects at/near 100% on its first pass across many independent items, that's itself a signal worth surfacing (to the model or the user) before burning further rounds.
Happy to share the full workflow script and journal if useful — redacted of anything sensitive.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the Workflow tool, Ultracode guidance, and the reported /workflows status while reproducing the supplied 12-role scenario. Use the offered workflow script and journal to trace agent-call growth, all-fail verification rounds, and user visibility; done means the behavior surfaces proportionate verification and non-convergence before further work is dispatched.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- markdown
- Domain
- ai, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100