Native Gemini routing for arbitrarily-named external endpoints: carry a profile key through chat params
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 5
- Forks
- 2
- Avg merge
- 13h 33m
- Merged PRs (30d)
- 35
Description
Follow-up to #61 (Databricks native APIs), which defers this.
Problem
Databricks native Gemini routing (google-generative protocol stamp) is restricted to endpoints whose name positively identifies the thinking variant (hosted pay-per-token names like databricks-gemini-2-5-pro). This is because ModelClientChatParams carries only model + thinkingEffort — at chat time, GeminiGenerateContentClient must re-derive the variant profile (budget vs level thinking control, ranges, off-ability) from the model string via ai-config's getGeminiGenerateContentProfile.
An external Gemini endpoint named e.g. my-gemini-endpoint backed by gemini-2.5-flash is classified correctly at discovery (the classifier sees external_model.name), but the client would have no way to reconstruct the variant from the endpoint name, so discovery deliberately stamps openai-chat instead (the classifier and client share one helper so stamp and wire choice cannot disagree).
Proposed direction
Widen the chat-params seam so the catalog pipeline can carry a discovery-time profile key (e.g. the resolved underlying model identity or a variant/profile id) into ModelClientChatParams. The client would prefer the carried key and fall back to name-derivation. Cross-repo change: bridge type plus catalog pipeline plumbing in consumers.
Scope notes
- Only worth doing if external Gemini endpoints on Databricks (or similar gateways) matter to users; hosted pay-per-token Gemini already routes natively.
- Could share the seam proposed in #62 (per-protocol capability provenance).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ModelClientChatParams, GeminiGenerateContentClient, and ai-config's getGeminiGenerateContentProfile. Trace the catalog classifier and consumer pipeline to identify where the discovery-time profile key can cross the bridge type. Done means external Gemini endpoints can retain native routing while name-based derivation remains the fallback, with the related cross-repo plumbing covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100