deepseek-ai / deepseek-ai/DeepEP

Questions about the Necessity of NVIDIA Driver Config Parameters

Open
#484 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

I have a question regarding the necessity of specific NVIDIA configuration parameters in the current version of DeepEP.

`options nvidia NVreg_EnableStreamMemOPs=1 NVreg_RegistryDwords="PeerMappingOverride=1;"`

However, in our tests, after removing these two parameters from initramfs, we still observe the following outputs:
```
NIC buffer will be on GPU memory
NIC handler will be GPU
```

From my observations, the version of `cuStreamWaitValue64` used in `nvshmem` is `11070_v2`, as indicated by:
```
LOAD_SYM(table, cuStreamWriteValue64, 11070, _v2, 1);
LOAD_SYM(table, cuStreamWaitValue64, 11070, _v2, 1);
```
According to the NVIDIA CUDA Driver API documentation, the `NVreg_EnableStreamMemOPs=1` parameter is only required for `v1` of these functions.

Additionally, does the `PeerMappingOverride` parameter only take effect in the case of non-root users?

So, are the parameters `NVreg_EnableStreamMemOPs=1` and `NVreg_RegistryDwords="PeerMappingOverride=1;"` still necessary in the current DeepEP?

What impact (if any) would there be without these parameters—for instance, on performance?

Thank you!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.