deepseek-ai / deepseek-ai/DeepEP

[Question] Question on `cpy_src_int4_ptr + offset_int4`

Closed
#525 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Cuda
Stars
10.1k
Forks
1.4k
Avg merge
2d 17h
Merged PRs (30d)
3

Description

First, I would like to express my great admiration for the authors who have created such an excellent project.

About the code in csrc/kernels/internode_ll.cu, I have a question regarding line 870's `cpy_src_int4_ptr + offset_int4`. First, `tma_load_1d`
(or `tma_load_and_arrive`) is used to transfer `kNumTMABufferBytes` bytes of data from global
memory to shared memory for the current warp's `next_stage_idx` in a single TMA operation,
where `kNumTMABufferBytes` represents the int4 data for 32 threads (one warp) for one stage,
with each lane responsible for `kNumSendUnrolls` int4s. However, in `offset_int4 = i + 32 *
kNumSendUnrolls`, `i` is lane-dependent (specifically, `i = lane_id * kNumSendUnrolls` at the
start of the loop). If `elect_one_sync()` at line 867 selects a thread in the warp that is
not lane=0, for example lane=2, then `cpy_src_int4_ptr + offset_int4` was intended to offset
to the next stage's data for 32 threads (i.e., `32 * kNumSendUnrolls` int4s), but it would
skip the `kNumSendUnrolls` int4s corresponding to lanes 0 and 1. How should this be
understood? Why is `(i + 32 * kNumSendUnrolls)` used to represent the data for
`next_stage_idx` that needs to be written from global memory to shared memory?

My personal humble opinion is that line 869 should perhaps be changed to the following to be more reasonable:
```cpp
const auto& offset_int4 = (iter_idx + 1) * 32 * kNumSendUnrolls;
```

I believe I must be misunderstanding something, but I cannot figure it out. I very much look forward to your reply. Thank you very much.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in csrc/kernels/internode_ll.cu around lines 867-870 and trace elect_one_sync(), lane_id, i, iter_idx, and next_stage_idx through the surrounding TMA load loop. Compare the pointer offset with the buffer layout and related tests or kernel checks; done means the offset behavior is explained or a confirmed defect is documented with a reproducible validation.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.