huggingface / huggingface/picotron
Expert parallelism
Open
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 203
- PR merge metrics
- No merged PRs in 30d
Description
Would be great to see a pico implementation of Expert parallelism as well! Thanks for the great repo.
Any reason we don't use something like ray here?
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named in the issue. Start by mapping the repository's existing parallelism implementation and determining how expert parallelism should fit; done means a pico implementation exists and the question of using Ray has been resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100