OpenHands / OpenHands/software-agent-sdk

Type LLM streaming, message, and tokenizer boundaries

Open
#4,976 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement ready-for-dev sdk typing
Dominant language
Python
Stars
1.1k
Forks
539
Avg merge
1d 19h
Merged PRs (30d)
137

Description

Parent tracking issue: #4904

Desired Behavior

LLM streaming wrappers, provider messages, tokenizer outputs, optional transformer loading, and response events should be represented by explicit protocols or adapter functions. Core LLM logic should consume those typed contracts with direct access instead of probing provider objects dynamically.

The fix in #4772 remains authoritative for the yielded-versus-wrapper completed-response bug; this work should build on or avoid conflicting with it.

Scope

  • Streaming/event and tokenizer access in llm.py
  • Provider message normalization in message.py and mixins/non_native_fc.py
  • OpenAI auth capability access
  • Excludes telemetry normalization and retry metadata

Acceptance Criteria

  • Streaming completion wrappers and response events have explicit typed contracts.
  • Tokenizer and optional-transformer capabilities are isolated behind typed adapters.
  • Provider message reasoning/thinking fields are normalized once before core message conversion.
  • Tests cover sync/async streams, missing optional capabilities, and supported provider message shapes.
  • Changes do not regress or duplicate #4772 and the baseline shrinks for removed calls.

This issue was created by an AI agent (OpenHands) on behalf of the user.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading llm.py, message.py, and mixins/non_native_fc.py, then review #4904 and the authoritative fix in #4772 to understand the boundaries and avoid duplication. Trace the existing streaming, tokenizer, optional-transformer, authentication, and provider-message access before defining the typed contracts. Done means the listed acceptance criteria pass, including sync/async and missing-capability tests, without telemetry or retry-metadata changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend-api-design
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.