Hmbown / Hmbown/Codewhale

Provider-neutral upstream vendor selection (per-model pin + discovery, beyond OpenRouter)

Open
#6,029 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
41k
Forks
3.6k
Avg merge
13h 59m
Merged PRs (30d)
299

Description

## Problem

Pinning an upstream vendor works today, but only for OpenRouter and only through config. `[providers.openrouter] vendor = "deepinfra"` emits `"provider": {"order": ["deepinfra"], "allow_fallbacks": false}` on every OpenRouter request path (`apply_openrouter_vendor`, `crates/config/src/lib.rs:164`).

Two gaps remain, and they are the same gap:

1. **It is OpenRouter-shaped.** OpenRouter is not the only aggregator that routes to upstream vendors. A `Vendor` pane wired to OpenRouter's `/models/{id}/endpoints` — Option C as originally scoped in #6007 — would be a surface we have to unpick the moment the second aggregator needs it.
2. **The pin is per-provider, not per-model.** One global `vendor` applies to every model on the route. There is no way to say "this model from vendor X, that model from vendor Y".

Maintainer intent, from #6007: *"Great points - going to think about how to best incorporate this across all of the multi vendor providers."*

## Scope

A provider-neutral way to express and select an upstream vendor:

- A vendor concept that is not OpenRouter-specific, with per-aggregator discovery of available vendors rather than one hardcoded endpoint shape.
- Per-`(model, vendor)` persistence, replacing the single route-wide `vendor` key without breaking it.
- A picker surface that shows the pin, joining the existing `/model` ordering rather than adding a parallel focus path.
- Whether `:vendor`-suffixed rows should appear in discovery views (#6007 open question 2).

## Explicitly out of scope

Per-vendor pricing. `docs/CONFIGURATION.md:504-505` states we do not fetch per-vendor endpoint prices, and a pinned route reports a routing-dependent unknown cost rather than a catalog estimate. Reporting a per-vendor rate we did not fetch would be inventing a receipt. If that changes it needs its own issue and a real source.

## Origin

Split out of #6007 (reported by @7jrxt42BxFZo4iAnN4CX), whose Options A and B shipped in `f7b5c412` / `f5755a88`. This carries the deferred Option C forward in a provider-neutral form so the scope is not lost when #6007 closes.

Contributor guide

Open the contributing guide

Research direction

Start with apply_openrouter_vendor and crates/config/src/lib.rs:164, then trace the existing /model ordering and the paths shipped in f7b5c412 and f5755a88 from #6007. Done means vendor selection is provider-neutral, persists per model and vendor without breaking the existing key, and appears in the picker; resolve how discovery and :vendor rows should work without adding per-vendor pricing.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design, cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.