agentscope-ai / agentscope-ai/QwenPaw
[Feature Request] Add Model Fallback Chain for LLM Provider Resilience
- Linguagem predominante
- Python
- Estrelas
- 34.9k
- Forks
- 3.1k
- Merge médio
- 1d 15h
- PRs com merge (30d)
- 225
Descrição
## Summary
Currently, QwenPaw only retries the **same** model on transient errors via `RetryChatModel`, and routes between local/cloud slots via `RoutingChatModel`. There is no mechanism to **automatically fall back to a different model or provider** when the primary one is unavailable, rate-limited, or returns persistent errors.
## Use Case
In production deployments (especially in regions with limited LLM API access, or when running expensive models like `qwen3-max` or `deepseek-v4-pro`), a single point of failure is fragile:
- A provider may return 429 rate-limits that exceed `LLM_MAX_RETRIES`.
- A model may be temporarily down for maintenance.
- Network issues may isolate one provider (e.g., `api.deepseek.com`) while another (e.g., DashScope, ModelScope) is still reachable.
A fallback chain would let the agent degrade gracefully: try primary -> try secondary -> try tertiary, before failing.
## Proposed Solution
Add a `fallback_models` field to the agent config and a `ChainChatModel` wrapper in `qwenpaw/agents/`:
```json
{
"agents": [{
"id": "default",
"active_model": {
"provider_id": "deepseek",
"model": "deepseek-v4-pro"
},
"fallback_models": [
{"provider_id": "dashscope", "model": "qwen3-max"},
{"provider_id": "modelscope", "model": "Qwen/Qwen3.5-122B-A10B"},
{"provider_id": "deepseek", "model": "deepseek-v4-flash"}
]
}]
}
```
### Sketch
```python
class ChainChatModel(ChatModelBase):
def __init__(self, primary: ChatModelBase, fallbacks: list[ChatModelBase]):
self.primary = primary
self.fallbacks = fallbacks
async def __call__(self, messages, **kwargs):
for i, model in enumerate([self.primary, *self.fallbacks]):
try:
return await model(messages, **kwargs)
except (RateLimitError, APIConnectionError, TimeoutException) as e:
logger.warning(f"ChainChatModel: model {i} failed, falling back. Error: {e}")
continue
raise
```
## Benefits
1. **Resilience**: Production agents survive transient provider outages.
2. **Cost optimization**: Chain from expensive to cheaper models (e.g., `qwen3-max` -> `qwen3-235b-a22b-thinking-2507` -> `qwen3.5-plus`).
3. **Geographic flexibility**: Mix domestic (DashScope) and international (DeepSeek, OpenRouter) providers.
4. **Composable**: Should work alongside the existing `RetryChatModel` (retry within a model, fallback across models).
## Related Code
- `qwenpaw/providers/retry_chat_model.py` - same-model retry (existing)
- `qwenpaw/agents/routing_chat_model.py` - local/cloud pre-routing (existing, not failure-based)
- `qwenpaw/agents/model_factory.py` - where chain could be wired
- `qwenpaw/agents/acp/server.py:_switch_model` - manual model switching (existing)
## Environment
- QwenPaw 1.1.9
- Default agent: `deepseek-v4-pro`
- WebUI: `http://127.0.0.1:8088`
---
Happy to submit a PR if the design direction is agreed on. Please share any constraints on the API shape.
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.