OpenHands / OpenHands/software-agent-sdk

Normalize LLM usage telemetry through a typed adapter

Open
#4,975 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement ready-for-dev sdk typing
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Parent tracking issue: #4904

Desired Behavior

LLM telemetry should normalize provider-specific completion and Responses API usage objects once at a typed boundary, then compute metrics from one internal usage representation using direct attribute access. Provider aliases such as prompt/input tokens, completion/output tokens, token details, cache reads/writes, and hidden response metadata should not be rediscovered repeatedly with getattr.

Scope

  • openhands-sdk/openhands/sdk/llm/utils/telemetry.py
  • Focused internal usage/response adapter types and telemetry tests
  • Dynamic-attribute baseline updates for this slice only

Acceptance Criteria

  • A typed internal usage representation covers completion and Responses API token fields.
  • Provider-shape normalization occurs once at the external response boundary.
  • Sync and async telemetry paths reuse the same normalized implementation.
  • Tests cover prompt/input, completion/output, reasoning, cache-read, cache-write, and missing-usage cases.
  • Telemetry's forbidden dynamic-attribute baseline entries are removed and relevant tests/type checks pass.

This issue was created by an AI agent (OpenHands) on behalf of the user.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with openhands-sdk/openhands/sdk/llm/utils/telemetry.py and the existing telemetry tests. Trace both sync and async paths and identify where provider-specific usage shapes are currently inspected. Done means a shared typed usage representation, boundary normalization, coverage for the listed token and missing-usage cases, and passing relevant tests and type checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.