agentscope-ai / agentscope-ai/QwenPaw
[Feature]: Automatic provider degradation similar to cc-switch to handle token quota exhaustion
- Langage dominant
- Python
- Étoiles
- 34.9k
- Forks
- 3.1k
- Merge moyen
- 1 j 15 h
- PR mergées (30 j)
- 225
Description
## Summary
Add a built-in automatic provider fallback mechanism so that agents can seamlessly switch to a backup LLM provider when the primary one hits token/rate limits, eliminating the need for manual configuration changes.
## Component(s) Affected
- [x] Core / Backend (app, agents, config, providers, utils, local_models)
- [ ] Console (frontend web UI)
- [ ] Channels (DingTalk, Feishu, QQ, Discord, iMessage, etc.)
- [ ] Skills
- [ ] CLI
- [ ] Documentation (website)
- [ ] Tests
- [ ] CI/CD
- [ ] Scripts / Deploy
## Problem / Motivation
When running long-lived agents or batch processing with providers like DashScope or OpenAI, quota exhaustion or rate limiting causes the agent to crash. The current workaround is to manually edit the provider configuration and restart the agent. This is unacceptable for:
- Autonomous agents that run for extended periods
- Production pipelines where manual intervention is impractical
- Multi-provider setups where users want automatic resilience
An automatic degradation pattern would keep agents running reliably without constant human supervision.
## Proposed Solution
Introduce a **prioritized fallback chain** with circuit-breaker semantics. The agent would accept a list of providers/models, each with a priority and optional cooldown. When the active provider fails with a retryable error (e.g., `rate_limit_exceeded`, `insufficient_quota`, 5xx), the system automatically routes requests to the next available provider.
Key behaviors:
- **Error classification**: only degrade on quota/rate/transient errors, not on fatal errors like invalid API keys.
- **Cooldown & retry**: higher-priority providers are periodically retried (with a test call) after a configurable cooldown, promoting them back automatically when recovered.
- **Transparent to agents**: the fallback logic lives in the model invocation layer so agents remain unchanged.
- **Configuration as code**:
```yaml
fallback_chain:
- provider: dashscope
model: qwen-max
priority: 1
cooldown: 300
- provider: openai
model: gpt-4o
priority: 2
- provider: zhipuai
model: glm-4
priority: 3
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.