agentscope-ai / agentscope-ai/QwenPaw
[Feature]: multi models adaptive routing
- Lenguaje dominante
- Python
- Estrellas
- 34.9k
- Forks
- 3.1k
- Merge medio
- 1 d 15 h
- PR fusionados (30 d)
- 225
Descripción
## Summary
Add support for multi-model switching to enable task-based model routing, allowing complex tasks to use strong models while simple tasks use lightweight models for token efficiency.
## Component(s) Affected
- [x] Core / Backend (app, agents, config, providers, utils, local_models)
- [ ] Console (frontend web UI)
- [ ] Channels (DingTalk, Feishu, QQ, Discord, iMessage, etc.)
- [ ] Skills
- [ ] CLI
- [ ] Documentation (website)
- [ ] Tests
- [ ] CI/CD
- [ ] Scripts / Deploy
## Problem / Motivation
Currently, CoPaw uses a single active LLM model for all tasks.
This leads to:
- Token Waste: Simple tasks (e.g., formatting, basic queries) consume expensive tokens from strong models
- Cost Inefficiency: Users pay premium rates for tasks that lightweight models could handle
- No Flexibility: Cannot leverage both cloud APIs and local models simultaneously
## Proposed Solution
### 1. Extend Model Slot Configuration
```python
# providers/models.py
class ModelSlotConfig(BaseModel):
provider_id: str = ""
model: str = ""
tier: str = "default" # "lightweight", "standard", "strong"
class ProvidersData(BaseModel):
providers: Dict[str, ProviderSettings] = {}
custom_providers: Dict[str, CustomProviderData] = {}
active_llm: ModelSlotConfig = ModelSlotConfig() # Keep for backward compat
model_slots: Dict[str, ModelSlotConfig] = { # New: multi-model slots
"lightweight": ModelSlotConfig(),
"standard": ModelSlotConfig(),
"strong": ModelSlotConfig(),
}
```
### 2. Task Complexity Classifier
```python
# agents/task_router.py
class TaskComplexity:
SIMPLE = "lightweight" # Formatting, short queries, simple tool calls
MODERATE = "standard" # Code review, file operations, basic reasoning
COMPLEX = "strong" # Multi-step reasoning, code generation, analysis
class TaskRouter:
def classify_task(self, query: str, tools: list) -> str:
# Classify based on query length, complexity indicators, tool requirements
...
```
### 3. Dynamic Model Factory
```python
# agents/model_factory.py
class ModelManager:
_instances: Dict[str, Tuple[ChatModelBase, FormatterBase]] = {}
@classmethod
def get_model_for_task(cls, complexity: str) -> Tuple[ChatModelBase, FormatterBase]:
slot = get_model_slot(complexity)
if slot.key not in cls._instances:
cls._instances[slot.key] = create_model_and_formatter(slot.config)
return cls._instances[slot.key]
```
### 4. Agent Integration
```python
# agents/react_agent.py
async def reply(self, msg: Msg) -> Msg:
complexity = self.task_router.classify_task(msg.content, self.toolkit.tools)
model, formatter = ModelManager.get_model_for_task(complexity)
# Use selected model for this task
```
## Alternatives Considered
- Manual Model Switching: Users manually switch models via CLI before each task
Drawback: Disrupts workflow, requires user intervention
- Two-Agent Architecture: Separate lightweight and strong agents
Drawback: Increased complexity, state synchronization issues
- Cost-Based Routing: Route based on API cost thresholds
Drawback: Doesn't account for task complexity, only price
## Additional Context
https://velvetshark.com/openclaw-multi-model-routing
## Willing to Contribute
- [x] I am willing to open a PR for this feature (after discussion).
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.