stacklok / stacklok/mecatl

OpenRouter provider pinning: adapter Option to send the provider routing preferences object (order / allow_fallbacks)

Open
#345 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Go
Stars
152
Forks
16
Avg merge
14h 48m
Merged PRs (30d)
536

Description

Why

#344 showed OpenRouter's multi-upstream load-balancing breaking OpenAI's stateless reasoning-replay contract: a reasoning encrypted_content blob minted by one upstream (e.g. OpenAI direct) is rejected by another (Azure) with invalid_encrypted_content, permanently bricking the session. The same heterogeneity family produced the Azure Duplicate item found with id fc_N bug and the Moonshot empty-text rejection (PR #291).

The prevention half is pinning the route to a single upstream so every turn of a multi-turn conversation is served by the party that can decrypt its own blobs. OpenRouter supports this via a request-body provider object:

"provider": { "order": ["openai"], "allow_fallbacks": false }

mecatl's request builder (internal/adapter/openai/request.go) doesn't send it today, so an operator has no way to opt out of cross-upstream routing.

Shape

  • An adapter-construction Option (like reasoning-effort / StreamIdleTimeout), NOT a port.LLMRequest field — LLMRequest is provider-neutral and must stay so (port/llm_neutral_test.go tripwires widening).
  • The openai-go SDK request would carry it via extra body fields; only the OpenRouter-flavored composition wires it (an OpenAI-direct or Azure-direct deployment must never send it).
  • Config surface: a provider-level key in settings.yaml (e.g. under the openrouter provider entry: provider_order: [...] / allow_fallbacks: false), threaded through internal/cliconfig / internal/app composition. Operator-tier config, per the posture of other provider knobs.
  • Values pass through verbatim/opaque — never validated against a hardcoded upstream list (forward-compat, same discipline as ProviderPhase).
  • Docs: user-docs/ deployment page for OpenRouter should mention that multi-turn reasoning models are safest pinned to one upstream, with a link to #344's failure mode.

Non-goals

  • No per-session or per-turn routing control; this is a deployment-level knob.
  • No attempt to auto-detect "sticky" routing — OpenRouter offers no such guarantee without the provider object.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in internal/adapter/openai/request.go, then trace adapter construction through internal/cliconfig and internal/app, using settings.yaml and port/llm_neutral_test.go to understand the configuration boundary. Check the user-docs/ deployment page for the OpenRouter section. Done means the provider routing object can be configured for OpenRouter only, values remain opaque, neutral requests stay unchanged, and the deployment guidance is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.