AlibabaResearch / AlibabaResearch/DAMO-ConvAI

[EvoTrainer] Any plan to support ms-swift as a training backend?

Open
#230 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.6k
Forks
250
PR merge metrics
No merged PRs in 30d

Description

Thanks for open-sourcing EvoTrainer. The `diagnosis → revision → backtesting → skill accumulation` loop and `skills/ANALYSIS_PLAYBOOK.md` are the most concrete write-up of an autonomous RL-tuning harness I have come across.

**Question:** is there any plan to support [ms-swift](https://github.com/modelscope/ms-swift) as a training backend in addition to ROLL?

**Context:** we run GRPO on ms-swift (`swift rlhf --rlhf_type grpo --use_vllm --vllm_mode colocate`) and would like to reuse EvoTrainer's harness rather than rebuild the diagnosis loop from scratch. From reading the tree, `roll/pipeline/agentic/` and its `env_manager/` look fairly tied to ROLL.

If there is no such plan, a few questions that would help us evaluate porting it ourselves:

**1. How framework-agnostic are the diagnostic dimensions?**
`skills/harness/agentic-rl-harness.md` describes reading rollout-level pkl files and TensorBoard events. Is that pkl schema documented and stable, or would a third-party backend need to match it exactly? Put differently: is there a minimal *evidence contract* — just the fields the rule engine actually consumes — that is smaller than ROLL's full dump?

**2. Where is the seam for the harness itself?**
The harness is described as reward composites + group filtering (e.g. `StdGroupFilter` with EMA threshold) + agent interaction constraints (`max_steps`, sequence length). Are those reachable through a defined interface that another trainer could implement, or are they inlined in the ROLL pipeline?

**3. Are the accumulated skills portable across GRPO implementations?**
Rules such as the Dead Group Ratio threshold and the Echo Trap / turn-inflation trigger appear to depend on GRPO group semantics rather than on ROLL specifically. Would you expect `skills/` to transfer to any GRPO implementation, or are the thresholds calibrated against ROLL's defaults?

Happy to contribute a swift adapter if the seams already exist and you would take a PR.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.