deepspeedai / deepspeedai/DeepSpeed
[Question] Difference between launch_bias_add_transform_0213 and launch_apply_rotary_pos_emb in ds_softmax_context
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
I'm confused about these codes.
Both func launch_bias_add_transform_0213 and launch_apply_rotary_pos_emb seems to do rotary_pos，what is the difference, why dose it need to execute rotary_pos twice when rotary_dim > 0 && rotate_half?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with csrc/transformer/inference/csrc/pt_binding.cpp at lines 480-510 and compare the calls to launch_bias_add_transform_0213 and launch_apply_rotary_pos_emb. Trace the rotary_dim > 0 && rotate_half path to determine each function's role and why both are invoked. Done means documenting the distinction and explaining whether rotary position processing is duplicated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100