MoonshotAI / MoonshotAI/FlashKDA
Add standalone usage example and inference-only note
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 1.3k
- Forks
- 122
- PR merge metrics
- No merged PRs in 30d
Description
Problem
The README only shows integration via FLA's chunk_kda interface. There is no standalone example showing direct usage of the flash_kda.fwd API, and no minimal reproduction script.
Recommendation
Add examples/basic_usage.py:
import torch
from flash_kda import fwd
B, H, T, D = 2, 8, 1024, 128
q = torch.randn(B, H, T, D, dtype=torch.bfloat16, device='cuda')
k = torch.randn_like(q)
v = torch.randn_like(q)
g = torch.randn(B, H, T, dtype=torch.float32, device='cuda')
beta = torch.rand(B, H, T, dtype=torch.float32, device='cuda').sigmoid()
scale = D ** -0.5
out = fwd(q, k, v, g, beta, scale)
print(f"Output shape: {out.shape}") # [2, 8, 1024, 128]
Also add a note in README that this is inference-only (no backward pass) — users needing training should fall back to FLA's Triton path.
Impact
Documentation. Reduces friction for new users who want to try the kernel directly without understanding FLA's internals.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the README and the existing FLA integration to match project conventions. Add examples/basic_usage.py using the documented flash_kda.fwd API, then add the inference-only note to the README, including the fallback to FLA's Triton path for training. Done means the standalone example is present and the README clearly explains the limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 85/100