[QST] StreamK ReductionStrategy: "Atomic" or "Mixed"

What is your question?

Hi, I'm learning/going through the StreamK implementation in CUTLASS, and came across various reduction strategies:

  /// Reduction strategy
  enum ReductionStrategy
  {
    kNone,      // Data-parallel strategy (no seams, fixup, etc.)

    kAtomic,    // Non-deterministic reduction of SK-block partials using atomic aggregation in L2

    kMixed,     // Deterministic reduction of SK-block partials employing either:
                //   (a) A separate wave of reduction thread blocks" (for scenarios with lots of
                //       SK-blocks per SK-tile)
                //   (b) Turnstile-ordered atomic aggregation in L2 (for scenarios with few
                //       SK-blocks per SK-tile)
  };

  static ReductionStrategy const kReductionStrategy = kMixed;

The current implementation hard-coded the choice. Does that mean the Mixed choice is strictly preferred to Atomic? It'd be great if someone could comment on situations when one strategy would be preferred to another --- thanks!!

Start with include/cutlass/gemm/threadblock/threadblock_swizzle_streamk.h, especially the hard-coded kReductionStrategy and the ReductionStrategy comments. Trace the StreamK reduction paths for kAtomic and kMixed and identify when each is selected or beneficial. Done means documenting the trade-offs and usage conditions clearly enough to answer the question.