feat(provider): support bounded auxiliary stream output ceilings
@JAORMX is already working on this.
Since Sep 7, 2026.
- Dominant language
- Go
- Stars
- 152
- Forks
- 16
- Avg merge
- 14h 48m
- Merged PRs (30d)
- 536
Description
Problem
Server-owned auxiliary model calls, beginning with session-title generation, may need an enforceable output-token ceiling before a provider starts generation. The current title generator validates streamed usage after completion, so its 128-token threshold detects/rejects an oversized response but cannot prevent the provider from generating and billing those tokens.
port.LLMRequest must remain provider-neutral and must not grow provider-specific controls. Today the title generator receives an already constructed port.LLMProvider, so it has no composition/provider-construction seam through which to bind a provider-native output limit.
Requested capability
Design a reusable, composition-owned capability for bounded auxiliary streams that can apply a provider-native output limit at request construction without widening port.LLMRequest or exposing routing/credentials to callers. Preserve the stream-side usage check as defense in depth.
Initial consumer
internal/adapter/server/title_generator.go requests a bounded title response (128 output tokens) but currently checks the limit only after ChunkUsage.
Scope notes
- This is a correctness/cost-containment guard for pathological or misbehaving output, not a claim that normal title calls will exceed the limit.
- The solution should be reusable by future host-owned auxiliary calls.
- Keep provider selection, credentials, and provider-private configuration in composition/adapter construction; do not widen the core request DTO.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.