stacklok / stacklok/mecatl

feat(provider): support bounded auxiliary stream output ceilings

Open
#1,160 0 comments 0 reactions 1 assignee View on GitHub

@JAORMX is already working on this.

Since Sep 7, 2026.

Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Problem

Server-owned auxiliary model calls, beginning with session-title generation, may need an enforceable output-token ceiling before a provider starts generation. The current title generator validates streamed usage after completion, so its 128-token threshold detects/rejects an oversized response but cannot prevent the provider from generating and billing those tokens.

port.LLMRequest must remain provider-neutral and must not grow provider-specific controls. Today the title generator receives an already constructed port.LLMProvider, so it has no composition/provider-construction seam through which to bind a provider-native output limit.

Requested capability

Design a reusable, composition-owned capability for bounded auxiliary streams that can apply a provider-native output limit at request construction without widening port.LLMRequest or exposing routing/credentials to callers. Preserve the stream-side usage check as defense in depth.

Initial consumer

internal/adapter/server/title_generator.go requests a bounded title response (128 output tokens) but currently checks the limit only after ChunkUsage.

Scope notes

  • This is a correctness/cost-containment guard for pathological or misbehaving output, not a claim that normal title calls will exceed the limit.
  • The solution should be reusable by future host-owned auxiliary calls.
  • Keep provider selection, credentials, and provider-private configuration in composition/adapter construction; do not widen the core request DTO.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.