NVIDIA / NVIDIA/cutile-python

[FEA]: Require ct.barrier for multi stage kernels

Open
#37 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request priority: P1 status: triaged
Dominant language
Python
Stars
2.2k
Forks
155
PR merge metrics
No merged PRs in 30d

Description

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request?

High

Please provide a clear description of problem this feature solves

In CUDA programming, we use atomic methods or cooperative groups to synchronize execution across blocks.
cutile could provide a similar mechanism to help developers write complex multi-stage kernels in a simpler way.

Feature Description

Example:

import torch
import cuda.tile as ct

@ct.kernel
def device_norm(
    x: ct.Array, y: ct.Array, workspace: ct.Array, 
    tile_size: ct.Constant, p: ct.Constant):
    # create a barrier on global memory, except p blocks to reach it.
    barrier = ct.barrier(p=p)
    block_id = ct.bid(0)
    
    tile = ct.load(x, index=(block_id, 0), shape=(1, tile_size))
    mean = ct.sum(tile) / tile_size
    
    ct.atomic_add(workspace, (0, ), mean)
    # wait until p blocks to reach here
    barrier.wait()

    global_mean = ct.load(workspace, (0, ), (1, ))
    global_mean = global_mean / p
    tile = tile - global_mean
    
    ct.store(y, (block_id, ), (tile_size, ))
Describe your ideal solution

Provide ct.barrier, or a similar feature, to make it easier for developers to write applications that require block-level synchronization.

There are multiple ways to implement ct.barrier:

  1. Allocate a region in global memory for synchronization, and let each block atomically increment a counter when it reaches the barrier.
  2. Use cooperative groups.
Describe any alternatives you have considered

No response

Additional context

No response

Contributing Guidelines
  • I agree to follow cuTile Python's contributing guidelines
  • I have searched the open feature requests and have found no duplicates for this feature request

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing ct.kernel, ct.load, ct.atomic_add, and ct.store entry points, then determine how a proposed ct.barrier would coordinate blocks across multi-stage kernels. Compare the global-memory counter and cooperative-groups approaches described in the issue. Done means a documented barrier feature supports the example synchronization flow and has validation for its semantics.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
hpc
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.