OpenHands / OpenHands/software-agent-sdk
[Bug]: ACP derived cost ignores cache/thought buckets, silently zeroes unknown models, and can inflate cost when UsageUpdate cost stalls
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 539
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 137
Description
Is there an existing issue for the same bug?
- I have searched existing issues and this is not a duplicate.
Bug Description
For ACP (external agent) sessions where the provider does not send UsageUpdate.cost (e.g. gemini-cli; the code comment at openhands/sdk/agent/acp_agent.py says "gemini-cli: does not send UsageUpdate (cost derived from tokens below)"), cost is derived from token counts. The derivation prices only input and output tokens, ignoring cache_read/cache_write and thought buckets. It also returns 0.0 silently for models unknown to litellm, and there is a path where a token-derived estimate is added even when provider-reported cost exists, inflating accumulated_cost.
Mechanism, verified at software-agent-sdk main (ecf417c), file openhands-sdk/openhands/sdk/agent/acp_agent.py:
_extract_token_usage(751-770) parses input/output/cache_read/cache_write/thought from ACP usage. The token metrics DO record these buckets._estimate_cost_from_tokens(777-793) prices onlyinput_tokens * input_cost + output_tokens * output_cost. The cache/thought buckets are never passed to it (signature is(model, input_tokens, output_tokens)). Cache reads are priced differently from plain input on most providers (cheaper) and cache writes at a premium, so a cache-heavy ACP session is mis-priced when this path runs.- Unknown models:
cost_map.get(model, {})yields an empty dict, both prices are 0, and the function returns 0.0. The blanketexcept Exception: return 0.0also swallows litellm import failures. The caller doesif cost > 0: add_cost(cost), so the silent result is that the turn records no cost at all, with no warning or metric. - Inflated-cost path: in
_record_usage, when aUsageUpdatearrives withcostbutdelta = cost.amount - last_cost <= 0(stale_last_cost_by_sessionafter a session switch, or a cost reset),cost_recordedstays False. The later branchif not cost_recorded and (input_tokens or output_tokens) and self.acp_model:then adds the token-derived estimate on top of the provider cost that was already recorded in an earlier call (theadd_cost(delta)sites at ~1815 and ~1846). Sessionaccumulated_costcan therefore exceed the provider's last cumulative cost. - Adjacent: failed/cancelled turns (timeout, hard-fail,
stop_reason=cancelled) typically never call_finalize_successful_turn/_record_usage, and a pending UsageUpdate can be dropped on the nextprepare_usage_sync.
Expected Behavior
- The derived cost should price cache_read/cache_write and thought tokens when the ACP usage reports them (litellm exposes
cache_read_input_token_cost,cache_creation_input_token_cost, and reasoning rates). - A token-derived estimate should only apply when no provider cost was recorded for the session, so the two sources cannot both contribute.
- An unknown model should not silently record $0; at minimum a warning or metric should surface it.
Steps To Reproduce
Static code-path demonstration (no live run required):
- Configure an ACP agent with a provider that sends usage without
UsageUpdate.cost(gemini-cli class). - Read
_record_usageinacp_agent.py(1792-1864): the provider-cost branch setscost_recorded = Trueonly whendelta > 0; the derived-cost branch fires whennot cost_recorded, regardless of whether provider cost was recorded in an earlier call. - For the cache/thought gap: pass a usage with nonzero
cache_read/cache_write/thoughtand observe_estimate_cost_from_tokens(model, input_tokens, output_tokens)never receives them.
Environment
- Python 3.12, macOS 15
- openhands-sdk software-agent-sdk main (ecf417c)
Installation Method
pip install from source (software-agent-sdk main, ecf417c)
SDK Version
ecf417c
Operating System
macOS 15 (arm64)
Additional Context
Suggested fix:
- Extend
_estimate_cost_from_tokensto accept and price cache_read/cache_write/thought buckets using litellm's cost fields. - Track a per-session "provider cost seen" flag independent of the last-delta, so the derived branch only runs when the provider never reported cost.
- When a model is unknown to litellm, emit a warning or record an unknown-cost marker instead of silently skipping.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in openhands-sdk/openhands/sdk/agent/acp_agent.py by reading _extract_token_usage, _estimate_cost_from_tokens, and _record_usage, especially the provider-cost and derived-cost branches. Trace the ACP usage buckets and per-session cost state; the work is done when cache/thought costs are included, provider and derived costs cannot double-count, and unknown models no longer silently record zero.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 64/100