agentscope-ai / agentscope-ai/QwenPaw

[Feature]: multi models adaptive routing

Abierto
#52 3 comentarios 1 reacción 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
Python
Estrellas
34.9k
Forks
3.1k
Merge medio
1 d 15 h
PR fusionados (30 d)
225

Descripción

## Summary

Add support for multi-model switching to enable task-based model routing, allowing complex tasks to use strong models while simple tasks use lightweight models for token efficiency.

## Component(s) Affected

- [x] Core / Backend (app, agents, config, providers, utils, local_models)
- [ ] Console (frontend web UI)
- [ ] Channels (DingTalk, Feishu, QQ, Discord, iMessage, etc.)
- [ ] Skills
- [ ] CLI
- [ ] Documentation (website)
- [ ] Tests
- [ ] CI/CD
- [ ] Scripts / Deploy

## Problem / Motivation

Currently, CoPaw uses a single active LLM model for all tasks.
This leads to:
- Token Waste: Simple tasks (e.g., formatting, basic queries) consume expensive tokens from strong models
- Cost Inefficiency: Users pay premium rates for tasks that lightweight models could handle
- No Flexibility: Cannot leverage both cloud APIs and local models simultaneously

## Proposed Solution
### 1. Extend Model Slot Configuration
```python
# providers/models.py
class ModelSlotConfig(BaseModel):
provider_id: str = ""
model: str = ""
tier: str = "default" # "lightweight", "standard", "strong"

class ProvidersData(BaseModel):
providers: Dict[str, ProviderSettings] = {}
custom_providers: Dict[str, CustomProviderData] = {}
active_llm: ModelSlotConfig = ModelSlotConfig() # Keep for backward compat
model_slots: Dict[str, ModelSlotConfig] = { # New: multi-model slots
"lightweight": ModelSlotConfig(),
"standard": ModelSlotConfig(),
"strong": ModelSlotConfig(),
}
```
### 2. Task Complexity Classifier
```python
# agents/task_router.py
class TaskComplexity:
SIMPLE = "lightweight" # Formatting, short queries, simple tool calls
MODERATE = "standard" # Code review, file operations, basic reasoning
COMPLEX = "strong" # Multi-step reasoning, code generation, analysis

class TaskRouter:
def classify_task(self, query: str, tools: list) -> str:
# Classify based on query length, complexity indicators, tool requirements
...
```
### 3. Dynamic Model Factory
```python
# agents/model_factory.py
class ModelManager:
_instances: Dict[str, Tuple[ChatModelBase, FormatterBase]] = {}

@classmethod
def get_model_for_task(cls, complexity: str) -> Tuple[ChatModelBase, FormatterBase]:
slot = get_model_slot(complexity)
if slot.key not in cls._instances:
cls._instances[slot.key] = create_model_and_formatter(slot.config)
return cls._instances[slot.key]
```
### 4. Agent Integration
```python
# agents/react_agent.py
async def reply(self, msg: Msg) -> Msg:
complexity = self.task_router.classify_task(msg.content, self.toolkit.tools)
model, formatter = ModelManager.get_model_for_task(complexity)
# Use selected model for this task
```
## Alternatives Considered

- Manual Model Switching: Users manually switch models via CLI before each task
Drawback: Disrupts workflow, requires user intervention
- Two-Agent Architecture: Separate lightweight and strong agents
Drawback: Increased complexity, state synchronization issues
- Cost-Based Routing: Route based on API cost thresholds
Drawback: Doesn't account for task complexity, only price

## Additional Context

https://velvetshark.com/openclaw-multi-model-routing

## Willing to Contribute

- [x] I am willing to open a PR for this feature (after discussion).

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.