AlibabaResearch / AlibabaResearch/DAMO-ConvAI

[EvoTrainer] Any plan to support ms-swift as a training backend?

Abierto
#230 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
1.6k
Forks
250
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Thanks for open-sourcing EvoTrainer. The `diagnosis → revision → backtesting → skill accumulation` loop and `skills/ANALYSIS_PLAYBOOK.md` are the most concrete write-up of an autonomous RL-tuning harness I have come across.

**Question:** is there any plan to support [ms-swift](https://github.com/modelscope/ms-swift) as a training backend in addition to ROLL?

**Context:** we run GRPO on ms-swift (`swift rlhf --rlhf_type grpo --use_vllm --vllm_mode colocate`) and would like to reuse EvoTrainer's harness rather than rebuild the diagnosis loop from scratch. From reading the tree, `roll/pipeline/agentic/` and its `env_manager/` look fairly tied to ROLL.

If there is no such plan, a few questions that would help us evaluate porting it ourselves:

**1. How framework-agnostic are the diagnostic dimensions?**
`skills/harness/agentic-rl-harness.md` describes reading rollout-level pkl files and TensorBoard events. Is that pkl schema documented and stable, or would a third-party backend need to match it exactly? Put differently: is there a minimal *evidence contract* — just the fields the rule engine actually consumes — that is smaller than ROLL's full dump?

**2. Where is the seam for the harness itself?**
The harness is described as reward composites + group filtering (e.g. `StdGroupFilter` with EMA threshold) + agent interaction constraints (`max_steps`, sequence length). Are those reachable through a defined interface that another trainer could implement, or are they inlined in the ROLL pipeline?

**3. Are the accumulated skills portable across GRPO implementations?**
Rules such as the Dead Group Ratio threshold and the Echo Trap / turn-inflation trigger appear to depend on GRPO group semantics rather than on ROLL specifically. Would you expect `skills/` to transfer to any GRPO implementation, or are the thresholds calibrated against ROLL's defaults?

Happy to contribute a swift adapter if the seams already exist and you would take a PR.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.