HarperFast / HarperFast/harper

[Agent Loop] Streaming token/cost budgets (maxToolTokens/maxCostUsd on generateStream)

Open
#854 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

Part of #612.

Enforce `maxToolTokens` / `maxCostUsd` on the **streaming** auto path (`generateStream`). Today these throw `501` at stream entry; the sync path enforces them.

## Why deferred

`GenerateChunk` doesn't expose `usage` in v1, so cumulative spend isn't observable from a stream. Wiring this needs the chunk type extended plus each in-tree backend (openai / anthropic / bedrock) emitting usage on its final chunk.

## Acceptance

- [ ] `GenerateChunk` (or a terminal metadata chunk) carries `usage`.
- [ ] Backends populate it on the stream's final chunk.
- [ ] Streaming loop tallies cumulative usage and trips `BudgetExceededError` like the sync path.
- [ ] Tests: streaming token-budget trip and cost-budget trip.

Note: the `maxToolIterations` cap already applies to streaming; this closes the spend-budget gap only.

---

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.