agentscope-ai / agentscope-ai/QwenPaw

[Feature]: Configuration refactoring: separate global/agent scopes, versioning, and comparison playground

未关闭
#4,758 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
enhancement
主要语言
Python
星标
34.9k
派生
3.1k
平均合并
1 天 15 小时
30 天内合并 PR
225

描述

## Summary

Refactor the current configuration system to clearly separate global settings from agent-level settings, introduce versioning for agent configurations, and provide a playground to compare the behavior of two different configurations side by side.

## Component(s) Affected

- [x] Core / Backend (app, agents, config, providers, utils, local_models)
- [x] Console (frontend web UI)
- [ ] Channels (DingTalk, Feishu, QQ, Discord, iMessage, etc.)
- [ ] Skills
- [ ] CLI
- [ ] Documentation (website)
- [ ] Tests
- [ ] CI/CD
- [ ] Scripts / Deploy

## Problem / Motivation

The current configuration has become disorganized:

- **Scope ambiguity**: Some settings are global (e.g., default providers, logging), while others are agent‑specific (e.g., system prompts, model parameters, tools). They are often mixed, making it hard to understand inheritance and overrides.
- **No versioning**: Agent configurations evolve rapidly during experimentation. Without built‑in versioning, users resort to manual file copies or git, leading to lost history, accidental overwrites, and difficulty in reproducing past behavior.
- **No comparison tool**: When tweaking prompts or parameters, users cannot easily quantify the impact—side‑by‑side output comparison is essential for prompt engineering and model selection.

This causes friction in development, debugging, and collaboration. Teams managing multiple agents or iterating on prompts need robust configuration management and a feedback loop to safely experiment.

## Proposed Solution

### 1. Configuration Structure Overhaul

Introduce a clean, two‑tier configuration model:

- **Global configuration** (`config/global.yaml` or similar) for settings that apply across all agents: default provider list, logging, telemetry, timeouts, etc.
- **Agent configuration** (one file per agent or stored in a database) for agent‑specific settings: `name`, `description`, `prompt_template`, `model`, `parameters`, `tool_bindings`, etc.

Provide clear documentation and validation so that agent files explicitly override global defaults where intended. The system should merge configurations predictably (e.g., agent‑level `model` overrides global `model`).

### 2. Agent Configuration Versioning

For each agent configuration, support:

- Automatic version snapshots on save, with a timestamp and an optional user‑provided description (e.g., “Added chain‑of‑thought instructions”).
- The ability to list, view, and restore previous versions.
- A diff view (textual or visual) to see what changed between two versions.

This could be backed by a simple file‑system history (`.versions/` subdirectory) or integrated into a lightweight database.

### 3. Configuration Comparison Playground

Add a new “Playground” tab in the Console (or a CLI command) that lets you:

- Select two agent configurations (any combination of versions, or the live version vs. a saved version).
- Input a common test message or multiple test cases.
- Run both configurations side by side and display their outputs, latency, token usage, and any errors.
- Highlight differences in the outputs (text diff) to easily assess quality changes.

This playground lowers the barrier to safe experimentation and encourages iterative improvement without risking the production agent.

## Alternatives Considered

- **Manual versioning with external version control (git)**: Works but requires context switching, lacks tight integration with the platform, and no built‑in comparison UI.
- **Copying configuration blocks and commenting out**: Error‑prone and quickly becomes unmanageable.
- **External A/B testing frameworks**: Overkill for simple prompt comparison, adds latency and complexity.

## Additional Context

Configuration management and prompt iteration are core workflows for AI agent development. Tools like LangSmith’s “Hub” and “Playground” have shown how valuable versioned configurations and side‑by‑side comparisons can be. Bringing these capabilities into QwenPaw would significantly improve the developer experience, especially for teams collaborating on agent behavior.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。