NVIDIA / NVIDIA/cutlass

[QST] Why we have both Ping-pong and Cooperative schedule?

Open
#2,181 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

? - Needs Triage inactive-30d inactive-90d question
Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Hi, I'm contemplating ideas around mainloop schedules of sm90 persistent kernels.

Cooperative and Ping-pong, both uses similar approach to hide the math ops from WGMMA.

As far as I know,

  • Cooperative: Split an output tile along M-dimension so we have two consumers dealing with upper and lower parts of same tile. Cooperative kernel can hide each other's epilogue and math ops. No order barrier.
  • Ping-pong: two consumer WGs work on different output tiles from scheduler. An epilogue op in one consumer is hidden by the mma op of another WG. This execution order is strictly guaranteed by a math order barrier.

My questions are:

  1. Why do ping-pong kernels strictly guarantee the execution order of two consumers?
  2. What's the rationale behind cooperative kernel have no order mechanism like above?
  3. Fp8 matmul kernels promote accumulator periodically. Can cooperative schedule hide the cost of the promotion? Apparently, ping-pong doesn't help in that case as it only overlaps epilogue with MMA.
  4. If cooperative kernel could hide the epilogue of other WGs, why do we have ping-pong schedule?

Thanks in advance!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file, test, or entry point is named. Start by locating the SM90 persistent-kernel implementations for the Cooperative and Ping-pong schedules, then trace their consumer work and barrier handling. Done would require a documented, evidence-based explanation addressing the four scheduling and FP8 promotion questions.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.