BOHICA-LABS / BOHICA-LABS/vsdd-factory

enhancement(orchestrator): provide token-budget visibility primitive (mirror Workflow's budget.spent/remaining)

Open
#220 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

The orchestrator has no quantitative visibility into the session's token cost. Auto Mode is binary (on/off). The orchestrator must make repeated "is one more burst worth it?" decisions at heuristic checkpoints without a budget signal — it asks the human via `AskUserQuestion` because there's no data-driven alternative.

The `Workflow` tool exposes a `budget.spent()` / `budget.remaining()` / `budget.total` primitive (per its tool description: "budget.total is null if no target was set; budget.spent() returns output tokens spent this turn"). The orchestrator agent does not have access to an equivalent.

## Reproduction

ftc-blue Phase 1d session (2026-06-23):

- Cumulative session use: estimated 4M+ tokens across ~35 agent dispatches over 9 adversarial passes.
- At three separate decision points (post-pass-8, post-pass-9, post-meta-analysis), the orchestrator paused to ask the human whether to continue. Each pause cost human-attention; each could have been data-driven with a budget signal.
- The orchestrator had no way to assess "we've spent X% of any allocated budget, recommend pausing" — only "we've done N passes, each pass costs ~600k tokens, here are the trade-offs."

## Proposed change

Provide the orchestrator with budget visibility:

**Option A — Tool primitive.**

Add a `BudgetGet` tool (or similar) to the orchestrator's tool profile that returns:

```
{
total: number | null, // session/turn budget if set
spent: number, // tokens consumed so far
remaining: number, // total - spent, or Infinity if unset
}
```

Mirrors the `budget` primitive in `Workflow`.

**Option B — System message injection.**

The harness periodically injects a system message with current token usage:

```
spent: 1.2M tokens / target: 5M tokens / 24% used
```

This is less precise (the orchestrator has to remember the most recent value) but requires no tool surface changes.

Option A is preferred for accuracy; Option B is simpler to ship.

## Applies to

- Agent tool framework / orchestrator dispatch internals
- `agents/orchestrator/AGENTS.md` — document how the orchestrator should react to budget signals (heuristics for "pause and ask" vs "continue autonomously")

## Acceptance criteria

- [ ] Orchestrator can query budget state programmatically (Option A) or sees it injected (Option B)
- [ ] AGENTS.md documents when to invoke budget heuristics (e.g., "if remaining < 1 burst-cost, pause and brief")
- [ ] Auto Mode behavior is tied to budget (e.g., "Auto Mode continues until budget < 20% remaining, then surfaces to human")

## Found during

ftc-blue Phase 1d adversarial cycle (2026-06-23, vsdd-factory@1.0.0-rc.21). Estimated 4M+ tokens used; no quantitative signal available to inform pacing decisions.

## Notes

The `Workflow` tool already has this. Mirroring it to the orchestrator's tool profile is a small, well-precedented change.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.