MoonshotAI / MoonshotAI/FlashKDA
Add Pythonic API wrapper with automatic workspace management
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 1.3k
- Forks
- 122
- PR merge metrics
- No merged PRs in 30d
Description
Problem
The C++ function requires an externally-managed workspace tensor, but the Python flash_kda/__init__.py is essentially empty — it just imports the C extension with no Pythonic wrapper.
Users currently need to:
size = flash_kda.get_workspace_size(B, H, T, CHUNK)
workspace = torch.empty(size, dtype=torch.uint8, device='cuda')
flash_kda.fwd(q, k, v, g, beta, scale, out, workspace=workspace, ...)
Recommendation
Add a Pythonic wrapper that handles workspace allocation automatically:
def fwd(q, k, v, g, beta, scale, out=None, **kwargs):
"""Forward pass for Kimi Delta Attention.
Args:
q: [B, H, T, D] query tensor (bf16/fp16)
k: [B, H, T, D] key tensor
v: [B, H, T, D] value tensor
g: [B, H, T] gate logits (fp32)
beta: [B, H, T] beta modulation (fp32)
scale: float, typically 1/sqrt(D)
Returns:
out: [B, H, T, D] output tensor
"""
if out is None:
out = torch.empty_like(q)
size = get_workspace_size(q.shape[0], q.shape[1], q.shape[2], 16)
workspace = torch.empty(size, dtype=torch.uint8, device=q.device)
_fwd_impl(q, k, v, g, beta, scale, out, workspace, **kwargs)
return out
Also add docstrings with expected shapes and dtypes, and define __all__ for clean from flash_kda import * behavior.
Impact
Usability. Makes the API accessible to users who don't want to manage CUDA workspace allocation manually. Reduces boilerplate and potential for OOM errors from incorrect size calculations.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in flash_kda/init.py and inspect the existing C extension entry points, including get_workspace_size and fwd. Add the documented Python wrapper, optional output allocation, automatic workspace allocation, and all; done means callers no longer need to create workspace tensors manually while the expected shapes and dtypes are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend-api-design, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100