NVIDIA / NVIDIA/cutlass

GemmWithKReduction parallel split-K: workspace initialization is skipped, partials land in the user output buffer

Open
#3,538 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CUTLASS C++
Dominant language
C++
Stars
10.5k
Forks
2.1k
Avg merge
3d 11h
Merged PRs (30d)
7

Description

Description

Since CUTLASS 3.2, GemmUniversalBase::initialize() only calls into Params::init_workspace when the mode is plain GEMM:

// include/cutlass/gemm/device/gemm_universal_base.h:415-420
// Assign and prepare workspace memory
if (args.mode == GemmUniversalMode::kGemm) {
  return params_.init_workspace(workspace, stream);
}
return Status::kSuccess;

GemmWithKReduction overrides Params::init_workspace to do its work exactly in the other mode:

// include/cutlass/gemm/kernel/gemm_with_k_reduction.h (init_workspace)
if (this->mode == GemmUniversalMode::kGemmSplitKParallel) {
  ptr_D = workspace;
  ptr_gemm_k_reduction = static_cast<uint8_t *>(workspace)
           + sizeof(ElementC) * size_t(this->batch_stride_D) * size_t(this->grid_tiled_shape.k());
  return Status::kSuccess;
}

The override can therefore never run: in kGemmSplitKParallel mode the base class skips the call entirely. Consequences for device::GemmWithKReduction with --parallel-split-k (example 23):

  1. The mainloop writes unreduced per-slice partial products through ptr_D, which still points at the user's D buffer; with a grid whose K extent exceeds the user D allocation this writes out of bounds.
  2. The separate reduction launch reads ptr_gemm_k_reduction from a never-initialized workspace.

get_workspace_size does reserve the right size in that mode, so the buffer exists and initialize() returns success; only the redirect is skipped.

Verified statically by reading both functions; additionally exercised on hardware via a probe subclass of the protected base (example 23 flow with --parallel-split-k): after a successful initialize(), params_.ptr_D still equals the user pointer while the reported workspace size is nonzero.

Suggested fix

Widen the gate in gemm_universal_base.h so the virtual init_workspace runs for every mode (the kernels' own overrides already no-op where nothing is needed), or special-case kGemmSplitKParallel alongside kGemm.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with include/cutlass/gemm/device/gemm_universal_base.h around the workspace initialization gate, then compare it with Params::init_workspace in include/cutlass/gemm/kernel/gemm_with_k_reduction.h. Exercise the example 23 --parallel-split-k flow or the reported probe and verify that initialization redirects partial output into workspace storage and the reduction reads initialized workspace data.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
70/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.