agentscope-ai / agentscope-ai/Trinity-RFT

[FEATURE] More granular config options for control of liger and optimizer

Open
#273 5 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
701
Forks
79
Avg merge
8h 7m
Merged PRs (30d)
1

Description

1. Would be nice to add `optimizer_fused`, `optimizer_foreach` config options. It allows to better control the memory footprint of the optimizer

Upstream issue:
- https://github.com/volcengine/verl/issues/2556

2. Would be nice to better control `_apply_liger_kernel_to_instance(model=actor_module)` ->
`_apply_liger_kernel_to_instance(model=actor_module, fused_linear_cross_entropy=liger_fused_linear_cross_entropy, cross_entropy=liger_cross_entropy)`. This is needed because otherwise Verl's fused FusedLinearCrossEntropy kernel conflicts with Liger kernels - at least used to conflict

Upstream issue:
- https://github.com/volcengine/verl/issues/2609

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.