MoonshotAI / MoonshotAI/FlashKDA

Partial output tiles serialize stores through one lane

Open
#14 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
1.3k
Forks
122
PR merge metrics
No merged PRs in 30d

Description

Problem

The recurrence kernel uses TMA for full 16-row output tiles, but falls back to a manual scalar copy for the final partial tile so it does not overwrite the next sequence.

That fallback currently executes only on the STORE warp lane selected by elect_one_sync(). With D=128, a tail of 15 rows therefore makes one lane perform:

15 rows * 128 columns = 1,920 scalar stores

The indexing is boundary-safe, but the entire tail copy is serialized.

Affected path

csrc/smxx/fwd_kernel2.cuh, in the actual_len < CHUNK output-store branch.

Desired outcome

  • Retain elected-lane ownership of output-pipeline wait and release operations.
  • Retain elected-lane ownership of full-tile and final-state TMA stores.
  • Distribute only the partial-tile scalar copy across the STORE warp.
  • Synchronize the warp before the shared-memory stage is released.
  • Cover tail lengths 1 through 15 at adjacent variable-length sequence boundaries.

A focused benchmark can be used to quantify the effect separately; the ownership and serialization issue is visible directly in the current kernel.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in csrc/smxx/fwd_kernel2.cuh at the actual_len < CHUNK output-store branch and inspect the existing STORE warp lane ownership and synchronization. Distribute only the partial-tile scalar copy across the STORE warp while preserving elected-lane pipeline and TMA operations. Done means tail lengths 1 through 15 are covered at adjacent variable-length sequence boundaries and the shared-memory stage is synchronized before release.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.