OpenHands / OpenHands/software-agent-sdk
Type LLM streaming, message, and tokenizer boundaries
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 539
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 137
Description
Parent tracking issue: #4904
Desired Behavior
LLM streaming wrappers, provider messages, tokenizer outputs, optional transformer loading, and response events should be represented by explicit protocols or adapter functions. Core LLM logic should consume those typed contracts with direct access instead of probing provider objects dynamically.
The fix in #4772 remains authoritative for the yielded-versus-wrapper completed-response bug; this work should build on or avoid conflicting with it.
Scope
- Streaming/event and tokenizer access in
llm.py - Provider message normalization in
message.pyandmixins/non_native_fc.py - OpenAI auth capability access
- Excludes telemetry normalization and retry metadata
Acceptance Criteria
- Streaming completion wrappers and response events have explicit typed contracts.
- Tokenizer and optional-transformer capabilities are isolated behind typed adapters.
- Provider message reasoning/thinking fields are normalized once before core message conversion.
- Tests cover sync/async streams, missing optional capabilities, and supported provider message shapes.
- Changes do not regress or duplicate #4772 and the baseline shrinks for removed calls.
This issue was created by an AI agent (OpenHands) on behalf of the user.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading llm.py, message.py, and mixins/non_native_fc.py, then review #4904 and the authoritative fix in #4772 to understand the boundaries and avoid duplication. Trace the existing streaming, tokenizer, optional-transformer, authentication, and provider-message access before defining the typed contracts. Done means the listed acceptance criteria pass, including sync/async and missing-capability tests, without telemetry or retry-metadata changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend-api-design
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100