MoonshotAI / MoonshotAI/FlashKDA
Partial output tiles serialize stores through one lane
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 1.3k
- Forks
- 122
- PR merge metrics
- No merged PRs in 30d
Description
Problem
The recurrence kernel uses TMA for full 16-row output tiles, but falls back to a manual scalar copy for the final partial tile so it does not overwrite the next sequence.
That fallback currently executes only on the STORE warp lane selected by elect_one_sync(). With D=128, a tail of 15 rows therefore makes one lane perform:
15 rows * 128 columns = 1,920 scalar stores
The indexing is boundary-safe, but the entire tail copy is serialized.
Affected path
csrc/smxx/fwd_kernel2.cuh, in the actual_len < CHUNK output-store branch.
Desired outcome
- Retain elected-lane ownership of output-pipeline wait and release operations.
- Retain elected-lane ownership of full-tile and final-state TMA stores.
- Distribute only the partial-tile scalar copy across the STORE warp.
- Synchronize the warp before the shared-memory stage is released.
- Cover tail lengths 1 through 15 at adjacent variable-length sequence boundaries.
A focused benchmark can be used to quantify the effect separately; the ownership and serialization issue is visible directly in the current kernel.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in csrc/smxx/fwd_kernel2.cuh at the actual_len < CHUNK output-store branch and inspect the existing STORE warp lane ownership and synchronization. Distribute only the partial-tile scalar copy across the STORE warp while preserving elected-lane pipeline and TMA operations. Done means tail lengths 1 through 15 are covered at adjacent variable-length sequence boundaries and the shared-memory stage is synchronized before release.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100