huggingface / huggingface/candle

Adapting llama_multiprocess to use `rsmpi`

Open
#904 6 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
21k
Forks
1.8k
Avg merge
16h 42m
Merged PRs (30d)
25

Description

Has anyone considered adapting `llama_multiprocess` to run on multiple machines instead of multiple processes? I've started by using the `SystemCommunicator` from `rsmpi` library to replace `nccl::Comm`, but the debugging seems pretty obscure. I managed to get it running but the last step hung forever. I can share more details.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the llama_multiprocess entry point and trace how nccl::Comm is used alongside rsmpi's SystemCommunicator. Reproduce the reported final-step hang and use the available issue details to determine what is needed for llama_multiprocess to complete across multiple machines.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.