huggingface / huggingface/candle
Adapting llama_multiprocess to use `rsmpi`
- Dominant language
- Rust
- Stars
- 21k
- Forks
- 1.8k
- Avg merge
- 16h 42m
- Merged PRs (30d)
- 25
Description
Has anyone considered adapting `llama_multiprocess` to run on multiple machines instead of multiple processes? I've started by using the `SystemCommunicator` from `rsmpi` library to replace `nccl::Comm`, but the debugging seems pretty obscure. I managed to get it running but the last step hung forever. I can share more details.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the llama_multiprocess entry point and trace how nccl::Comm is used alongside rsmpi's SystemCommunicator. Reproduce the reported final-step hang and use the available issue details to determine what is needed for llama_multiprocess to complete across multiple machines.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100