agentscope-ai / agentscope-ai/Trinity-RFT

[FEATURE] More granular config options for control of liger and optimizer

Aperta
#273 5 commenti 0 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
Python
Stelle
701
Fork
79
Merge medio
8h 7m
PR unite (30g)
1

Descrizione

1. Would be nice to add `optimizer_fused`, `optimizer_foreach` config options. It allows to better control the memory footprint of the optimizer

Upstream issue:
- https://github.com/volcengine/verl/issues/2556

2. Would be nice to better control `_apply_liger_kernel_to_instance(model=actor_module)` ->
`_apply_liger_kernel_to_instance(model=actor_module, fused_linear_cross_entropy=liger_fused_linear_cross_entropy, cross_entropy=liger_cross_entropy)`. This is needed because otherwise Verl's fused FusedLinearCrossEntropy kernel conflicts with Liger kernels - at least used to conflict

Upstream issue:
- https://github.com/volcengine/verl/issues/2609

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.