AlibabaResearch / AlibabaResearch/DAMO-ConvAI

[EvoTrainer] Any plan to support ms-swift as a training backend?

オープン
#230 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
1.6k
フォーク
250
PR マージ指標
30日以内にマージされた PR はありません

説明

Thanks for open-sourcing EvoTrainer. The `diagnosis → revision → backtesting → skill accumulation` loop and `skills/ANALYSIS_PLAYBOOK.md` are the most concrete write-up of an autonomous RL-tuning harness I have come across.

**Question:** is there any plan to support [ms-swift](https://github.com/modelscope/ms-swift) as a training backend in addition to ROLL?

**Context:** we run GRPO on ms-swift (`swift rlhf --rlhf_type grpo --use_vllm --vllm_mode colocate`) and would like to reuse EvoTrainer's harness rather than rebuild the diagnosis loop from scratch. From reading the tree, `roll/pipeline/agentic/` and its `env_manager/` look fairly tied to ROLL.

If there is no such plan, a few questions that would help us evaluate porting it ourselves:

**1. How framework-agnostic are the diagnostic dimensions?**
`skills/harness/agentic-rl-harness.md` describes reading rollout-level pkl files and TensorBoard events. Is that pkl schema documented and stable, or would a third-party backend need to match it exactly? Put differently: is there a minimal *evidence contract* — just the fields the rule engine actually consumes — that is smaller than ROLL's full dump?

**2. Where is the seam for the harness itself?**
The harness is described as reward composites + group filtering (e.g. `StdGroupFilter` with EMA threshold) + agent interaction constraints (`max_steps`, sequence length). Are those reachable through a defined interface that another trainer could implement, or are they inlined in the ROLL pipeline?

**3. Are the accumulated skills portable across GRPO implementations?**
Rules such as the Dead Group Ratio threshold and the Echo Trap / turn-inflation trigger appear to depend on GRPO group semantics rather than on ROLL specifically. Would you expect `skills/` to transfer to any GRPO implementation, or are the thresholds calibrated against ROLL's defaults?

Happy to contribute a swift adapter if the seams already exist and you would take a PR.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。