agentscope-ai / agentscope-ai/QwenPaw

[Feature]: multi models adaptive routing

Đang mở
#52 3 bình luận 1 reaction 0 người được giao Xem trên GitHub
enhancement
Ngôn ngữ chính
TypeScript
Star
35k
Fork
3.1k
Merge trung bình
1 ngày 13 giờ
Pull request đã merge (30 ngày)
228

Mô tả

## Summary

Add support for multi-model switching to enable task-based model routing, allowing complex tasks to use strong models while simple tasks use lightweight models for token efficiency.

## Component(s) Affected

- [x] Core / Backend (app, agents, config, providers, utils, local_models)
- [ ] Console (frontend web UI)
- [ ] Channels (DingTalk, Feishu, QQ, Discord, iMessage, etc.)
- [ ] Skills
- [ ] CLI
- [ ] Documentation (website)
- [ ] Tests
- [ ] CI/CD
- [ ] Scripts / Deploy

## Problem / Motivation

Currently, CoPaw uses a single active LLM model for all tasks.
This leads to:
- Token Waste: Simple tasks (e.g., formatting, basic queries) consume expensive tokens from strong models
- Cost Inefficiency: Users pay premium rates for tasks that lightweight models could handle
- No Flexibility: Cannot leverage both cloud APIs and local models simultaneously

## Proposed Solution
### 1. Extend Model Slot Configuration
```python
# providers/models.py
class ModelSlotConfig(BaseModel):
provider_id: str = ""
model: str = ""
tier: str = "default" # "lightweight", "standard", "strong"

class ProvidersData(BaseModel):
providers: Dict[str, ProviderSettings] = {}
custom_providers: Dict[str, CustomProviderData] = {}
active_llm: ModelSlotConfig = ModelSlotConfig() # Keep for backward compat
model_slots: Dict[str, ModelSlotConfig] = { # New: multi-model slots
"lightweight": ModelSlotConfig(),
"standard": ModelSlotConfig(),
"strong": ModelSlotConfig(),
}
```
### 2. Task Complexity Classifier
```python
# agents/task_router.py
class TaskComplexity:
SIMPLE = "lightweight" # Formatting, short queries, simple tool calls
MODERATE = "standard" # Code review, file operations, basic reasoning
COMPLEX = "strong" # Multi-step reasoning, code generation, analysis

class TaskRouter:
def classify_task(self, query: str, tools: list) -> str:
# Classify based on query length, complexity indicators, tool requirements
...
```
### 3. Dynamic Model Factory
```python
# agents/model_factory.py
class ModelManager:
_instances: Dict[str, Tuple[ChatModelBase, FormatterBase]] = {}

@classmethod
def get_model_for_task(cls, complexity: str) -> Tuple[ChatModelBase, FormatterBase]:
slot = get_model_slot(complexity)
if slot.key not in cls._instances:
cls._instances[slot.key] = create_model_and_formatter(slot.config)
return cls._instances[slot.key]
```
### 4. Agent Integration
```python
# agents/react_agent.py
async def reply(self, msg: Msg) -> Msg:
complexity = self.task_router.classify_task(msg.content, self.toolkit.tools)
model, formatter = ModelManager.get_model_for_task(complexity)
# Use selected model for this task
```
## Alternatives Considered

- Manual Model Switching: Users manually switch models via CLI before each task
Drawback: Disrupts workflow, requires user intervention
- Two-Agent Architecture: Separate lightweight and strong agents
Drawback: Increased complexity, state synchronization issues
- Cost-Based Routing: Route based on API cost thresholds
Drawback: Doesn't account for task complexity, only price

## Additional Context

https://velvetshark.com/openclaw-multi-model-routing

## Willing to Contribute

- [x] I am willing to open a PR for this feature (after discussion).

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.