agentscope-ai / agentscope-ai/QwenPaw

[Feature Request] Add Model Fallback Chain for LLM Provider Resilience

Abierto
#4,882 1 comentario 0 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
TypeScript
Estrellas
35k
Forks
3.1k
Merge medio
1 d 13 h
PR fusionados (30 d)
228

Descripción

## Summary

Currently, QwenPaw only retries the **same** model on transient errors via `RetryChatModel`, and routes between local/cloud slots via `RoutingChatModel`. There is no mechanism to **automatically fall back to a different model or provider** when the primary one is unavailable, rate-limited, or returns persistent errors.

## Use Case

In production deployments (especially in regions with limited LLM API access, or when running expensive models like `qwen3-max` or `deepseek-v4-pro`), a single point of failure is fragile:

- A provider may return 429 rate-limits that exceed `LLM_MAX_RETRIES`.
- A model may be temporarily down for maintenance.
- Network issues may isolate one provider (e.g., `api.deepseek.com`) while another (e.g., DashScope, ModelScope) is still reachable.

A fallback chain would let the agent degrade gracefully: try primary -> try secondary -> try tertiary, before failing.

## Proposed Solution

Add a `fallback_models` field to the agent config and a `ChainChatModel` wrapper in `qwenpaw/agents/`:

```json
{
"agents": [{
"id": "default",
"active_model": {
"provider_id": "deepseek",
"model": "deepseek-v4-pro"
},
"fallback_models": [
{"provider_id": "dashscope", "model": "qwen3-max"},
{"provider_id": "modelscope", "model": "Qwen/Qwen3.5-122B-A10B"},
{"provider_id": "deepseek", "model": "deepseek-v4-flash"}
]
}]
}
```

### Sketch

```python
class ChainChatModel(ChatModelBase):
def __init__(self, primary: ChatModelBase, fallbacks: list[ChatModelBase]):
self.primary = primary
self.fallbacks = fallbacks

async def __call__(self, messages, **kwargs):
for i, model in enumerate([self.primary, *self.fallbacks]):
try:
return await model(messages, **kwargs)
except (RateLimitError, APIConnectionError, TimeoutException) as e:
logger.warning(f"ChainChatModel: model {i} failed, falling back. Error: {e}")
continue
raise
```

## Benefits

1. **Resilience**: Production agents survive transient provider outages.
2. **Cost optimization**: Chain from expensive to cheaper models (e.g., `qwen3-max` -> `qwen3-235b-a22b-thinking-2507` -> `qwen3.5-plus`).
3. **Geographic flexibility**: Mix domestic (DashScope) and international (DeepSeek, OpenRouter) providers.
4. **Composable**: Should work alongside the existing `RetryChatModel` (retry within a model, fallback across models).

## Related Code

- `qwenpaw/providers/retry_chat_model.py` - same-model retry (existing)
- `qwenpaw/agents/routing_chat_model.py` - local/cloud pre-routing (existing, not failure-based)
- `qwenpaw/agents/model_factory.py` - where chain could be wired
- `qwenpaw/agents/acp/server.py:_switch_model` - manual model switching (existing)

## Environment

- QwenPaw 1.1.9
- Default agent: `deepseek-v4-pro`
- WebUI: `http://127.0.0.1:8088`

---

Happy to submit a PR if the design direction is agreed on. Please share any constraints on the API shape.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.