MoonshotAI / MoonshotAI/FlashKDA

Add Pythonic API wrapper with automatic workspace management

Open
#23 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
1.3k
Forks
122
PR merge metrics
No merged PRs in 30d

Description

Problem

The C++ function requires an externally-managed workspace tensor, but the Python flash_kda/__init__.py is essentially empty — it just imports the C extension with no Pythonic wrapper.

Users currently need to:

size = flash_kda.get_workspace_size(B, H, T, CHUNK)
workspace = torch.empty(size, dtype=torch.uint8, device='cuda')
flash_kda.fwd(q, k, v, g, beta, scale, out, workspace=workspace, ...)

Recommendation

Add a Pythonic wrapper that handles workspace allocation automatically:

def fwd(q, k, v, g, beta, scale, out=None, **kwargs):
    """Forward pass for Kimi Delta Attention.
    
    Args:
        q: [B, H, T, D] query tensor (bf16/fp16)
        k: [B, H, T, D] key tensor
        v: [B, H, T, D] value tensor
        g: [B, H, T] gate logits (fp32)
        beta: [B, H, T] beta modulation (fp32)
        scale: float, typically 1/sqrt(D)
    
    Returns:
        out: [B, H, T, D] output tensor
    """
    if out is None:
        out = torch.empty_like(q)
    size = get_workspace_size(q.shape[0], q.shape[1], q.shape[2], 16)
    workspace = torch.empty(size, dtype=torch.uint8, device=q.device)
    _fwd_impl(q, k, v, g, beta, scale, out, workspace, **kwargs)
    return out

Also add docstrings with expected shapes and dtypes, and define __all__ for clean from flash_kda import * behavior.

Impact

Usability. Makes the API accessible to users who don't want to manage CUDA workspace allocation manually. Reduces boilerplate and potential for OOM errors from incorrect size calculations.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in flash_kda/init.py and inspect the existing C extension entry points, including get_workspace_size and fwd. Add the documented Python wrapper, optional output allocation, automatic workspace allocation, and all; done means callers no longer need to create workspace tensors manually while the expected shapes and dtypes are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.