Better Custom Endpoint BYOK
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
1. Add auto-fetch models and most model settings.
2. Allow configuring Base URL for only one times.
3. Allow omitting most of the model configs.
4. Allow global model settings.
5. Add detailed information: icons, descriptions, total CWS, credits, and more.
See the example configuration file below for more information.
```jsonc
[
{
"name": "Custom Endpoint",
"vendor": "customendpoint",
"url": "https://example.com/api",
"apiType": "chat-completions",
/*
1. Allow configuring url outside `models`.
2. Automatically (responses > messages > chat-completions) use the available API type when `apiType` is not provided.
2. Allow full URL (e.g. https://example.com/v1/chat/completions). When using full URL, auto-detect `apiType` and ignore user's config.
*/
"autoFetch": true, // Add: Auto-fetch. If not configured, defaultly enable (only) when `models` is empty.
"global": { // Add: Global settings (optional). Can be overrided by per-model settings below.
"description": "A model with 1M context windows size and vision.", // Add: Model description. Shown when the cursor hovers on the model name in the picker.
"icon": "https://example.com/model.svg", // Add: Model icon. Shown in model picker. Available options: Icon URL, "claude", "gpt", "gemini", etc.
"toolCalling": true,
"vision": true,
"maxInputTokens": 1024000,
"maxOutputTokens": 64000,
"maxTokens": 1024000, // Add: Total context window size. Use the smaller one of configured CWS here and (`maxInputTokens` + `maxOutputTokens`).
"credits": {
// Add: Credits per 1M tokens.
"input": 1,
"output": 3,
"cacheRead": 0.1
}
},
"models": [
{
"id": "custom",
// Everything below is optional. If not provided, automatically fetch the information below if possible.
"name": "My Model",
"description": "A model with 1M context windows size and vision.",
"icon": "https://example.com/model.svg",
"toolCalling": true,
"vision": true,
"maxInputTokens": 1024000,
"maxOutputTokens": 64000,
"maxTokens": 1024000,
"credits": {
"input": 1,
"output": 3,
"cacheRead": 0.1
}
}
] // The model list itself is optional too--auto-fetch if empty.
}
]
```
Contributor guide
Assessment
This issue has not been assessed yet.