Feature: Support extraBody configuration for BYOK custom models
- Dominant language
- TypeScript
- Stars
- 193k
- Forks
- 42.4k
- PR merge metrics
- PR metrics pending
Description
## Summary
Allow BYOK (Bring Your Own Key) custom models to configure additional request body fields via an `extraBody` property in `chatLanguageModels.json`. This would enable users to pass model-specific parameters that are not currently supported through the existing `modelOptions` configuration.
(**This text was created with Copilot!**)
## Use Case
When using custom OpenAI-compatible endpoints (like vLLM, local inference servers, or specialized APIs), models often require additional parameters beyond the currently supported `temperature` and `top_p`:
- `reasoning_effort` (for reasoning models)
- `preserve_thinking` (to retain internal reasoning in responses)
- `chat_template_kwargs` (for custom chat template configuration)
- Provider-specific parameters for cache control, routing, or session management
- Custom model options (e.g., `output_config`, nested configuration objects)
### Example Configuration
```json
{
"vendor": "customendpoint",
"models": [
{
"id": "GLM-5.3-Flash-EXL3",
"name": "2xDGX-GLM-5.3-Flash-EXL3",
"url": "http://myAIServer:8600/v1",
"toolCalling": true,
"vision": true,
"maxInputTokens": 262144,
"maxOutputTokens": 32768,
"modelOptions": {
"temperature": 0.2,
"top_p": 0.95
},
"extraBody": {
"reasoning_effort": "none",
"preserve_thinking": false,
"chat_template_kwargs": {
"enable_thinking": false
}
}
}
]
}
```
## Current Limitations
1. **Only `temperature` and `top_p` are supported** in `modelOptions` (see `byokProvider.ts` lines 308-321 and `openAIEndpoint.ts` lines 302-324)
2. **No generic body customization** - users cannot pass arbitrary request body fields
3. **Model initialization failures** - models that require additional parameters cannot be properly configured
4. **Inconsistent with other extensions** - some extensions (e.g., Kilo Code) already support custom header and body configuration
## Proposed Solution
### Option A: Flexible `extraBody` Field (Recommended)
Add an optional `extraBody` field to model capabilities that allows arbitrary JSON to be merged into the request body:
```typescript
export interface BYOKModelCapabilities {
// ... existing fields ...
/**
* Additional fields to merge into the request body.
* These override default values but can be overridden by per-request parameters.
*/
extraBody?: Record;
}
```
**Implementation:**
- Merge `extraBody` into the request body after model options are applied
- Deep merge for nested objects
- Per-request parameters take precedence
### Option B: Explicit Parameter List
Extend `modelOptions` to support common additional parameters:
- `reasoning_effort`
- `preserve_thinking`
- Custom output configuration options
**Pros:** More type-safe, better validation
**Cons:** Less flexible, requires updating schema for each new parameter
## Additional Context
- Related to #316794 (custom headers request)
- Affects users of local deployments (vLLM, ollama, custom inference servers)
- Impacts compatibility with specialized reasoning models (DeepSeek, Claude with thinking)
- See #312746 for related issue with reasoning content handling
## Suggested Implementation Path
1. Add `extraBody?: Record` to `BYOKModelCapabilities` interface
2. Update JSON schema in `package.json` to allow `extraBody` object
3. Modify `OpenAIEndpoint._applyConfiguredModelOptions()` to merge `extraBody` into request body
4. Ensure per-request parameters still take precedence
5. Add unit tests for body merging behavior
6. Document in language model customization guide
## Related Issues
- #316794 - Support custom headers or stable chat session identifiers for BYOK
- #312746 - Copilot Chat drops reasoning_content field with DeepSeek models
- #319968 - Optional model discovery for customendpoint
- #318748 - Custom endpoint model display issues
Contributor guide
Assessment
This issue has not been assessed yet.