agentscope-ai / agentscope-ai/QwenPaw

[Feature Request] Add Model Fallback Chain for LLM Provider Resilience

Aberta
#4,882 1 comentário 0 reações 0 responsáveis Ver no GitHub
enhancement
Linguagem predominante
Python
Estrelas
34.9k
Forks
3.1k
Merge médio
1d 15h
PRs com merge (30d)
225

Descrição

## Summary

Currently, QwenPaw only retries the **same** model on transient errors via `RetryChatModel`, and routes between local/cloud slots via `RoutingChatModel`. There is no mechanism to **automatically fall back to a different model or provider** when the primary one is unavailable, rate-limited, or returns persistent errors.

## Use Case

In production deployments (especially in regions with limited LLM API access, or when running expensive models like `qwen3-max` or `deepseek-v4-pro`), a single point of failure is fragile:

- A provider may return 429 rate-limits that exceed `LLM_MAX_RETRIES`.
- A model may be temporarily down for maintenance.
- Network issues may isolate one provider (e.g., `api.deepseek.com`) while another (e.g., DashScope, ModelScope) is still reachable.

A fallback chain would let the agent degrade gracefully: try primary -> try secondary -> try tertiary, before failing.

## Proposed Solution

Add a `fallback_models` field to the agent config and a `ChainChatModel` wrapper in `qwenpaw/agents/`:

```json
{
"agents": [{
"id": "default",
"active_model": {
"provider_id": "deepseek",
"model": "deepseek-v4-pro"
},
"fallback_models": [
{"provider_id": "dashscope", "model": "qwen3-max"},
{"provider_id": "modelscope", "model": "Qwen/Qwen3.5-122B-A10B"},
{"provider_id": "deepseek", "model": "deepseek-v4-flash"}
]
}]
}
```

### Sketch

```python
class ChainChatModel(ChatModelBase):
def __init__(self, primary: ChatModelBase, fallbacks: list[ChatModelBase]):
self.primary = primary
self.fallbacks = fallbacks

async def __call__(self, messages, **kwargs):
for i, model in enumerate([self.primary, *self.fallbacks]):
try:
return await model(messages, **kwargs)
except (RateLimitError, APIConnectionError, TimeoutException) as e:
logger.warning(f"ChainChatModel: model {i} failed, falling back. Error: {e}")
continue
raise
```

## Benefits

1. **Resilience**: Production agents survive transient provider outages.
2. **Cost optimization**: Chain from expensive to cheaper models (e.g., `qwen3-max` -> `qwen3-235b-a22b-thinking-2507` -> `qwen3.5-plus`).
3. **Geographic flexibility**: Mix domestic (DashScope) and international (DeepSeek, OpenRouter) providers.
4. **Composable**: Should work alongside the existing `RetryChatModel` (retry within a model, fallback across models).

## Related Code

- `qwenpaw/providers/retry_chat_model.py` - same-model retry (existing)
- `qwenpaw/agents/routing_chat_model.py` - local/cloud pre-routing (existing, not failure-based)
- `qwenpaw/agents/model_factory.py` - where chain could be wired
- `qwenpaw/agents/acp/server.py:_switch_model` - manual model switching (existing)

## Environment

- QwenPaw 1.1.9
- Default agent: `deepseek-v4-pro`
- WebUI: `http://127.0.0.1:8088`

---

Happy to submit a PR if the design direction is agreed on. Please share any constraints on the API shape.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.