microsoft / microsoft/amplifier

validate-bundle: detect tool-task + delegate co-residency (silent ~27K token/request duplication)

Open
#376 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug dx token-cost validate-bundle
Dominant language
Python
Stars
3.1k
Forks
261
Avg merge
3h 28m
Merged PRs (30d)
13

Description

Problem

validate-bundle-repo.yaml and the bundle-design-expert do not currently detect when tool-task and the modern delegate tool are co-resident in the same bundle or behavior. When both are loaded in the same session, the full agent catalog is serialized into each tool's description field independently — doubling the per-request catalog cost with zero added capability.

Found in amplifier-bundle-tech-bro-idealist's behaviors/product-management.yaml, where - module: tool-task was declared alongside the foundation-provided delegate. Measured cost: ~27K tokens of redundant agent catalog per request — charged before the user types anything.

Root cause

tool-task (legacy) and tool-delegate (modern replacement) both independently serialize the full registered agent catalog into their tool description field. Foundation's behaviors document these as mutually exclusive, but nothing in the bundle validation or authoring workflow surfaces a warning when both are present in the same composition.

From Foundation docs: "task is the legacy task tool — for legacy task tool, use the tasks behavior instead." They are documented as not meant to be co-resident, but the documentation lives in a behavior file, not in a validator.

Impact

Every consumer of a behavior or bundle that declares tool-task and runs inside a host session that already provides delegate (i.e., any modern Amplifier session) pays the full duplicate catalog cost silently. At typical agent registry sizes this is ~27K extra tokens per request.

Tokens are charged per-request, not per-session — this compounds over every turn.

Evidence

  • amplifier-bundle-tech-bro-idealist behaviors/product-management.yaml had - module: tool-task on line 25 alongside foundation-provided delegate. Fixed in that repo (commit 34ccf3a).
  • amplifier_foundation/bundle_docs/token_cost.py already tracks all_tool_schemas with color thresholds — the infrastructure exists to surface this class of problem.

Requested changes

1. validate-bundle.yaml / validate-bundle-repo.yaml (in amplifier-foundation)
Add a check that flags tool-task and any delegate-equivalent tool as co-resident. Suggested failure message:

tool-task is the legacy agent-spawning module and is mutually exclusive with the delegate tool. Remove tool-task — PM agents are spawned by the host session's delegate tool.

2. bundle-design-expert guidance (in amplifier-foundation)
Add tool-task co-residency to the bundle anti-patterns knowledge. The expert should warn proactively during DESIGN and BUILD mode when an author adds tool-task to a bundle or behavior intended to compose into a modern Amplifier session.

3. token_cost.py thresholds (optional / bonus)
Consider whether duplicated agent-catalog tools should be surfaced as a distinct cost category in the token cost model, separate from legitimate tool schema cost.

Labels to add

bug, validate-bundle, token-cost, dx

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading validate-bundle.yaml and validate-bundle-repo.yaml, then inspect the bundle-design-expert guidance and bundle_docs/token_cost.py. Trace how tool modules are identified and how existing schema-cost checks are surfaced. Done means co-resident tool-task and delegate usage is flagged, the authoring guidance warns about it, and the optional token-cost treatment has a documented decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
developer-experience, performance, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.