deepseek-ai / deepseek-ai/DeepEP

some questions in ibgda

Open
#113 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
4d 1h
Merged PRs (30d)
2

Description

Hi, I have some questions about the following code in ibgda

```cpp
void ibgda_submit_requests(nvshmemi_ibgda_device_qp_t *qp, uint64_t base_wqe_idx,
uint32_t num_wqes, int message_idx = 0) {
nvshmemi_ibgda_device_qp_management_t *mvars = &qp->mvars;
uint64_t new_wqe_idx = base_wqe_idx + num_wqes;

// WQE writes must be finished first
__threadfence(); // (1)

// Wait for prior WQE slots to be filled first
auto *ready_idx = reinterpret_cast(&mvars->tx_wq.ready_head);
while (atomicCAS(ready_idx, base_wqe_idx, new_wqe_idx) != base_wqe_idx); // (2)

// Always post, not in batch
constexpr int kNumRequestInBatch = 4;
if (kAlwaysDoPostSend or (message_idx + 1) % kNumRequestInBatch == 0)
ibgda_post_send(qp, new_wqe_idx);
}
```

(1) I personally understand that the purpose of threadfence here is to ensure that writing to the WQE and writing to the DB do not occur out of order. From the view of the NIC, should threadfence_system be used instead?
(2) I personally understand that all threads executing atomicCAS have different compare/swap values, so is it necessary to use "atomic" operations in this case?

Thanks

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.