MoonshotAI / MoonshotAI/FlashKDA

Add standalone usage example and inference-only note

Open Beginner friendly
#21 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
1.3k
Forks
122
PR merge metrics
No merged PRs in 30d

Description

Problem

The README only shows integration via FLA's chunk_kda interface. There is no standalone example showing direct usage of the flash_kda.fwd API, and no minimal reproduction script.

Recommendation

Add examples/basic_usage.py:

import torch
from flash_kda import fwd

B, H, T, D = 2, 8, 1024, 128
q = torch.randn(B, H, T, D, dtype=torch.bfloat16, device='cuda')
k = torch.randn_like(q)
v = torch.randn_like(q)
g = torch.randn(B, H, T, dtype=torch.float32, device='cuda')
beta = torch.rand(B, H, T, dtype=torch.float32, device='cuda').sigmoid()
scale = D ** -0.5

out = fwd(q, k, v, g, beta, scale)
print(f"Output shape: {out.shape}")  # [2, 8, 1024, 128]

Also add a note in README that this is inference-only (no backward pass) — users needing training should fall back to FLA's Triton path.

Impact

Documentation. Reduces friction for new users who want to try the kernel directly without understanding FLA's internals.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the README and the existing FLA integration to match project conventions. Add examples/basic_usage.py using the documented flash_kda.fwd API, then add the inference-only note to the README, including the fallback to FLA's Triton path for training. Done means the standalone example is present and the README clearly explains the limitation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
85/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.