OpenHands / OpenHands/software-agent-sdk

[Bug]: ACP derived cost ignores cache/thought buckets, silently zeroes unknown models, and can inflate cost when UsageUpdate cost stalls

Open
#4,382 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

acp bug llm priority:medium
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Is there an existing issue for the same bug?

  • I have searched existing issues and this is not a duplicate.

Bug Description

For ACP (external agent) sessions where the provider does not send UsageUpdate.cost (e.g. gemini-cli; the code comment at openhands/sdk/agent/acp_agent.py says "gemini-cli: does not send UsageUpdate (cost derived from tokens below)"), cost is derived from token counts. The derivation prices only input and output tokens, ignoring cache_read/cache_write and thought buckets. It also returns 0.0 silently for models unknown to litellm, and there is a path where a token-derived estimate is added even when provider-reported cost exists, inflating accumulated_cost.

Mechanism, verified at software-agent-sdk main (ecf417c), file openhands-sdk/openhands/sdk/agent/acp_agent.py:

  1. _extract_token_usage (751-770) parses input/output/cache_read/cache_write/thought from ACP usage. The token metrics DO record these buckets.
  2. _estimate_cost_from_tokens (777-793) prices only input_tokens * input_cost + output_tokens * output_cost. The cache/thought buckets are never passed to it (signature is (model, input_tokens, output_tokens)). Cache reads are priced differently from plain input on most providers (cheaper) and cache writes at a premium, so a cache-heavy ACP session is mis-priced when this path runs.
  3. Unknown models: cost_map.get(model, {}) yields an empty dict, both prices are 0, and the function returns 0.0. The blanket except Exception: return 0.0 also swallows litellm import failures. The caller does if cost > 0: add_cost(cost), so the silent result is that the turn records no cost at all, with no warning or metric.
  4. Inflated-cost path: in _record_usage, when a UsageUpdate arrives with cost but delta = cost.amount - last_cost <= 0 (stale _last_cost_by_session after a session switch, or a cost reset), cost_recorded stays False. The later branch if not cost_recorded and (input_tokens or output_tokens) and self.acp_model: then adds the token-derived estimate on top of the provider cost that was already recorded in an earlier call (the add_cost(delta) sites at ~1815 and ~1846). Session accumulated_cost can therefore exceed the provider's last cumulative cost.
  5. Adjacent: failed/cancelled turns (timeout, hard-fail, stop_reason=cancelled) typically never call _finalize_successful_turn / _record_usage, and a pending UsageUpdate can be dropped on the next prepare_usage_sync.

Expected Behavior

  • The derived cost should price cache_read/cache_write and thought tokens when the ACP usage reports them (litellm exposes cache_read_input_token_cost, cache_creation_input_token_cost, and reasoning rates).
  • A token-derived estimate should only apply when no provider cost was recorded for the session, so the two sources cannot both contribute.
  • An unknown model should not silently record $0; at minimum a warning or metric should surface it.

Steps To Reproduce

Static code-path demonstration (no live run required):

  1. Configure an ACP agent with a provider that sends usage without UsageUpdate.cost (gemini-cli class).
  2. Read _record_usage in acp_agent.py (1792-1864): the provider-cost branch sets cost_recorded = True only when delta > 0; the derived-cost branch fires when not cost_recorded, regardless of whether provider cost was recorded in an earlier call.
  3. For the cache/thought gap: pass a usage with nonzero cache_read/cache_write/thought and observe _estimate_cost_from_tokens(model, input_tokens, output_tokens) never receives them.

Environment

  • Python 3.12, macOS 15
  • openhands-sdk software-agent-sdk main (ecf417c)

Installation Method

pip install from source (software-agent-sdk main, ecf417c)

SDK Version

ecf417c

Operating System

macOS 15 (arm64)

Additional Context

Suggested fix:

  1. Extend _estimate_cost_from_tokens to accept and price cache_read/cache_write/thought buckets using litellm's cost fields.
  2. Track a per-session "provider cost seen" flag independent of the last-delta, so the derived branch only runs when the provider never reported cost.
  3. When a model is unknown to litellm, emit a warning or record an unknown-cost marker instead of silently skipping.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in openhands-sdk/openhands/sdk/agent/acp_agent.py by reading _extract_token_usage, _estimate_cost_from_tokens, and _record_usage, especially the provider-cost and derived-cost branches. Trace the ACP usage buckets and per-session cost state; the work is done when cache/thought costs are included, provider and derived costs cannot double-count, and unknown models no longer silently record zero.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
64/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.