AlibabaResearch / AlibabaResearch/DAMO-ConvAI

[EvoTrainer] Any plan to support ms-swift as a training backend?

未关闭
#230 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
1.6k
派生
250
PR 合并指标
30 天内没有已合并 PR

描述

Thanks for open-sourcing EvoTrainer. The `diagnosis → revision → backtesting → skill accumulation` loop and `skills/ANALYSIS_PLAYBOOK.md` are the most concrete write-up of an autonomous RL-tuning harness I have come across.

**Question:** is there any plan to support [ms-swift](https://github.com/modelscope/ms-swift) as a training backend in addition to ROLL?

**Context:** we run GRPO on ms-swift (`swift rlhf --rlhf_type grpo --use_vllm --vllm_mode colocate`) and would like to reuse EvoTrainer's harness rather than rebuild the diagnosis loop from scratch. From reading the tree, `roll/pipeline/agentic/` and its `env_manager/` look fairly tied to ROLL.

If there is no such plan, a few questions that would help us evaluate porting it ourselves:

**1. How framework-agnostic are the diagnostic dimensions?**
`skills/harness/agentic-rl-harness.md` describes reading rollout-level pkl files and TensorBoard events. Is that pkl schema documented and stable, or would a third-party backend need to match it exactly? Put differently: is there a minimal *evidence contract* — just the fields the rule engine actually consumes — that is smaller than ROLL's full dump?

**2. Where is the seam for the harness itself?**
The harness is described as reward composites + group filtering (e.g. `StdGroupFilter` with EMA threshold) + agent interaction constraints (`max_steps`, sequence length). Are those reachable through a defined interface that another trainer could implement, or are they inlined in the ROLL pipeline?

**3. Are the accumulated skills portable across GRPO implementations?**
Rules such as the Dead Group Ratio threshold and the Echo Trap / turn-inflation trigger appear to depend on GRPO group semantics rather than on ROLL specifically. Would you expect `skills/` to transfer to any GRPO implementation, or are the thresholds calibrated against ROLL's defaults?

Happy to contribute a swift adapter if the seams already exist and you would take a PR.

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。