Make dynamic skill-menu failures deterministic across Anthropic and pi
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 197
- Forks
- 51
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 94
Description
Problem
Dynamic-skill programs depend on load_skill_menu, but a failed menu fetch produces different and model-dependent behavior across the two harnesses.
On the Anthropic path, createWizardToolsMcpServer prefetches skill-menu.json to build a Zod enum. When the fetch fails, the enum falls back to only integration. A valid call such as load_skill_menu { category: "feature-flags" } is then schema-rejected before the handler can return an actionable menu error.
On the pi path, the category is an unrestricted string, but a failed fetch returns the plain tool result Error: could not load the skill menu. Neither path is converted by the host into a terminal Wizard error. The agent must interpret the tool failure and emit the program's configured [ABORT] signal. That usually works when the prompt names this case, but it is not deterministic and is not covered by a failure-path test.
Why this matters
Programs without a fixed skillId, including ai-observability and feature-flags, cannot proceed without the menu. A transient registry outage should fail once with a useful explanation. It should not depend on the selected model to translate a schema error, retry, or eventually stop for another reason.
Relevant code
src/lib/wizard-tools/mcp.ts:157-169- failed prefetch leavescategoryNamesas['integration'].src/lib/wizard-tools/mcp.ts:340-369- Anthropic schema and handler.src/lib/agent/runner/harness/pi/tools.ts:86-112- pi fetch and error result.src/lib/agent/runner/sequence/linear.ts:142-263- terminal error routing has no skill-menu failure case.
Suggested direction
Keep the category input callable even when startup discovery fails, then represent registry unavailability explicitly. The exact implementation can stay shared-tool-level, but Anthropic and pi should expose the same failure semantics and the runner should not rely on agent wording to finish the run.
Acceptance
- A failed
skill-menu.jsonfetch does not narrow Anthropic's category schema to an unrelated fallback value. - Anthropic and pi return equivalent structured failures when the menu is unavailable.
- A dynamic-skill program terminates with one actionable error rather than relying on model retries or prose.
- Focused tests cover menu-fetch failure for both harnesses.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the failure paths in src/lib/wizard-tools/mcp.ts and src/lib/agent/runner/harness/pi/tools.ts, then trace terminal routing in src/lib/agent/runner/sequence/linear.ts. Run the focused harness tests if available and add coverage for failed skill-menu fetches in both Anthropic and pi. Done means both harnesses expose equivalent structured failures and a dynamic-skill run terminates with one actionable error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100