linkedin / linkedin/Liger-Kernel
Megatron Support
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.6k
- Forks
- 603
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 47
Description
### 🚀 The feature, motivation and pitch
Thank you for developing such an outstanding library. With its help, my pre-training work has achieved an MFU of 51%. Truly appreciate your library.
In larger-scale training scenarios, frameworks based on Megatron are commonly utilized for training. I was wondering if it would be possible to integrate Liger-kernel into Megatron or if you could provide relevant tutorials.
Thank you once again for your remarkable work!
### Alternatives
_No response_
### Additional context
_No response_
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. First clarify whether the goal is Megatron integration or tutorials, then identify the relevant Megatron and Liger-Kernel integration points. Completion criteria are not defined in the report and must be agreed before implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100