linkedin / linkedin/Liger-Kernel
Problem when running GRPO with liger kernel.
- Dominant language
- Python
- Stars
- 6.6k
- Forks
- 603
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 47
Description
Hi, thanks for your work.
When I run Qwen2.5-VL using GRPO with liger kernel in ms-swift, it happens:
But when I turn --use_liger_kernel into false, it can run correctly.
My version:
torch: 2.7.1
cuda: 128
transformers: 4.53.2
flash_attn: 2.8.1
liger kernel: 0.6.0
How to solve this? Dose it related to the LigerFusedLinearGRPOLoss? Thanks!
Contributor guide
Research direction
Start by reproducing Qwen2.5-VL with GRPO and --use_liger_kernel enabled using the listed versions, then compare the run with that option disabled. Investigate the LigerFusedLinearGRPOLoss entry point and use the failure shown in the attached image to identify the incompatibility; done means the configuration runs correctly or the limitation is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100