MoonshotAI / MoonshotAI/FlashKDA

[Question] Roadmap for supporting additional K/V head dimensions and backward kernels

Open
#4 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Cuda
Stars
1.3k
Forks
122
PR merge metrics
No merged PRs in 30d

Description

Hi, thanks for open-sourcing FlashKDA!

I am currently evaluating it as a backend for fla.ops.kda.chunk_kda.
From the README and API, my understanding is that the current implementation is mainly targeting inference-mode forward execution, and seems to assume K/V head dimension = 128.

I have a couple of questions regarding the roadmap:

  1. Are there any plans to support K/V head dimensions other than 128?
    For example, we have seen some KDA configurations where the per-head K/V dimension is 64 rather than 128. We are not specifically asking only about D=64, but more generally whether FlashKDA is expected to support additional common head dimensions in the future.
    If support for other dimensions is planned, would it likely be implemented as separate CUDA/CUTLASS specializations, or as a more general implementation?
  2. Are there any plans to add backward/autograd support, or is the project intended to remain an inference-only forward backend?

Thanks again for your work, and have a great weekend ahead!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the README and the API for fla.ops.kda.chunk_kda, then inspect how the current inference-mode forward path handles K/V head dimension 128. The issue does not name implementation files, tests, or a defined acceptance condition; completion would require an agreed roadmap or concrete scope for additional dimensions and backward/autograd support.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.