agentscope-ai / agentscope-ai/QwenPaw

[Feature]: Automatic provider degradation similar to cc-switch to handle token quota exhaustion

Ouverte
#4,757 2 commentaires 0 réactions 1 personne assignée Réclamée par @pan-x-c Voir sur GitHub
enhancement
Langage dominant
Python
Étoiles
34.9k
Forks
3.1k
Merge moyen
1 j 15 h
PR mergées (30 j)
225

Description

## Summary

Add a built-in automatic provider fallback mechanism so that agents can seamlessly switch to a backup LLM provider when the primary one hits token/rate limits, eliminating the need for manual configuration changes.

Image

## Component(s) Affected

- [x] Core / Backend (app, agents, config, providers, utils, local_models)
- [ ] Console (frontend web UI)
- [ ] Channels (DingTalk, Feishu, QQ, Discord, iMessage, etc.)
- [ ] Skills
- [ ] CLI
- [ ] Documentation (website)
- [ ] Tests
- [ ] CI/CD
- [ ] Scripts / Deploy

## Problem / Motivation

When running long-lived agents or batch processing with providers like DashScope or OpenAI, quota exhaustion or rate limiting causes the agent to crash. The current workaround is to manually edit the provider configuration and restart the agent. This is unacceptable for:

- Autonomous agents that run for extended periods
- Production pipelines where manual intervention is impractical
- Multi-provider setups where users want automatic resilience

An automatic degradation pattern would keep agents running reliably without constant human supervision.

## Proposed Solution

Introduce a **prioritized fallback chain** with circuit-breaker semantics. The agent would accept a list of providers/models, each with a priority and optional cooldown. When the active provider fails with a retryable error (e.g., `rate_limit_exceeded`, `insufficient_quota`, 5xx), the system automatically routes requests to the next available provider.

Key behaviors:
- **Error classification**: only degrade on quota/rate/transient errors, not on fatal errors like invalid API keys.
- **Cooldown & retry**: higher-priority providers are periodically retried (with a test call) after a configurable cooldown, promoting them back automatically when recovered.
- **Transparent to agents**: the fallback logic lives in the model invocation layer so agents remain unchanged.
- **Configuration as code**:

```yaml
fallback_chain:
- provider: dashscope
model: qwen-max
priority: 1
cooldown: 300
- provider: openai
model: gpt-4o
priority: 2
- provider: zhipuai
model: glm-4
priority: 3

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.