deepspeedai / deepspeedai/DeepSpeed
[GUIDE] NUMA Affinity Guide for small level of clusters
@tjruwase is already working on this.
Since Nov 20, 2021.
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
Is your feature request related to a problem? Please describe.
Memory movement speed does not matters when we increase number of nodes in Zero Infinity.
But it would be better to suggest NUMA affinity guide to increase memory copy speed up for various system configuration
For example, on our A100 pod using AMD EPYC 7543 32-Core Processor, NUMA affinity setting increase memcpy speed.
- We set NUMA affinity for A100 GPU that does not have interleaved memory on NUMA node.
- Memcpy speed for host to device memory increase up to 24GB/s which is BW peak of each socket lane.
- Without NUMA settings, variance of memcpy speed is quite large(4~12GB/s).
- Memcpy speed can limit between dual-socket lane.
Describe the solution you'd like
- It would be better to guide NUMA affinity settings for system configuration.
- Can DeepSpeed launcher can support NUMA affinity settings based on system information?
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.