feat(ai-proxy): add defaults field for fallback model parameters
- Dominant language
- Lua
- Stars
- 17.1k
- Forks
- 2.9k
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 63
Description
## Summary
Add a `defaults` field to the `ai-proxy` and `ai-proxy-multi` plugins that provides fallback model parameters applied only when the client request does not specify them.
## Motivation
Currently, the `options` field always overrides request values regardless of what the client sends. This is useful for enforcing parameters, but there is no way to provide sensible defaults that the client can still override.
A common use case is an LLM gateway where the operator wants to set default values (e.g., `max_tokens`, `temperature`) without restricting clients from customizing these values in their requests.
## Proposed Design
Add a `defaults` field alongside the existing `options` field:
```yaml
plugins:
ai-proxy:
model:
provider: openai
name: gpt-4o
defaults:
max_tokens: 512
temperature: 0.7
options:
model: gpt-4o # always enforced
```
**Priority order** (highest to lowest):
1. `options` — always overrides, even if the client sends a value
2. client request body — client-provided values
3. `defaults` — fallback, applied only when the client omits the field
## Difference from `options`
| Field | Behavior |
|-------|----------|
| `options` | Always overrides request values (enforcement) |
| `defaults` | Only applied when the client does not set the field (fallback) |
## Examples
Given `defaults: { max_tokens: 512, temperature: 0.7 }` and `options: { model: gpt-4o }`:
| Client sends | Effective value |
|-------------|-----------------|
| `max_tokens: 1024` | `1024` (client wins over defaults) |
| _(no max_tokens)_ | `512` (defaults applied) |
| `model: gpt-3.5-turbo` | `gpt-4o` (options always wins) |
## Implementation
A working implementation is available in PR #12895. Happy to revise based on community feedback.
Contributor guide
Research direction
Read the existing options handling in the ai-proxy and ai-proxy-multi plugins, then compare it with the working implementation in PR #12895. Verify that defaults apply only when the client omits a parameter, client values override defaults, and options still take precedence; cover these priority cases before considering the feature complete.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- lua
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100