agentscope-ai / agentscope-ai/Trinity-RFT

[FEATURE] More granular config options for control of liger and optimizer

Abierto
#273 5 comentarios 0 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
Python
Estrellas
701
Forks
79
Merge medio
8 h 7 min
PR fusionados (30 d)
1

Descripción

1. Would be nice to add `optimizer_fused`, `optimizer_foreach` config options. It allows to better control the memory footprint of the optimizer

Upstream issue:
- https://github.com/volcengine/verl/issues/2556

2. Would be nice to better control `_apply_liger_kernel_to_instance(model=actor_module)` ->
`_apply_liger_kernel_to_instance(model=actor_module, fused_linear_cross_entropy=liger_fused_linear_cross_entropy, cross_entropy=liger_cross_entropy)`. This is needed because otherwise Verl's fused FusedLinearCrossEntropy kernel conflicts with Liger kernels - at least used to conflict

Upstream issue:
- https://github.com/volcengine/verl/issues/2609

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.