deepseek-ai / deepseek-ai/EPLB

An example to help understanding how does EPLB work

Open
#12 1 comment 21 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
210
PR merge metrics
No merged PRs in 30d

Description

Base on the [example](https://github.com/deepseek-ai/EPLB/blob/main/README.md), if we want to reassign the 12 experts into 8 GPUs of 2 nodes for layer1, how to reassign experts to make balance for each GPU?
There are three steps to assign experts:
Step1: inter-node balance: Divide experts into 4 groups(0-2, 3-5, 6-8, 9-11), and assign the 4 groups into 2 nodes, ensuring inter-node balance, which can be seen as a backpack problem and solved using greedy algorithms.
Step2: expert balance: Replicate the hot experts, which within each node(4,5;1,10).
Step3: intra-node balance: Pack the replicated experts to individual GPUs(intra-node) to ensure different GPUs are load-balanced, which can alse be seen as a backpack problem.
The entire process can be called as **Hierarchical Load Balancing** as follow:
![Image](https://github.com/user-attachments/assets/600637ff-13f6-48ab-bf60-2d7fe82e5363)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.