Owner

Dao-AILab

6 indexed repositories · View on GitHub

flash-attention
Fast and memory-efficient exact attention

Python · 24976 stars
quack
A Quirky Assortment of CuTe Kernels

Python · 1146 stars
causal-conv1d
Causal depthwise conv1d in CUDA, with a PyTorch interface

Python · 953 stars
sonic-moe
Accelerating MoE with IO and Tile-aware Optimizations

Python · 769 stars
gram-newton-schulz
Fast Polar Decomposition for Muon

Python · 183 stars
gambit-parallel-reasoning
Thought-level beam search for parallel reasoning via hardware-aware pruning and adaptive compute reallocation

Python · 7 stars

Beginner friendly Everything

Open All

[SM90] [CuteDSL] Thoughts on adding support for QSA type attention? Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

Dao-AILab/flash-attention#2898 · Opened Sep 18, 2026
[CuTe, SM80] Forward silently ignores `block_sparse_tensors` and returns dense attention Open

Difficulty 2/5 1-3 hours Newbie friendliness 86/100

Dao-AILab/flash-attention#2897 · Opened Sep 18, 2026
[SM100] Missing PDL waits before grouped GEMM metadata reads Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

Dao-AILab/quack#206 · Opened Sep 16, 2026
Verify evals on Papers with Code Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Dao-AILab/gambit-parallel-reasoning#2 · Opened Sep 14, 2026
Cannot install on Windows - Aiter path too long Open

Difficulty 3/5 1-2 days Newbie friendliness 35/100

Dao-AILab/flash-attention#2861 · Opened Sep 5, 2026
[FA4][SM120] flash_attn_varlen_func / flash_attn_func illegal memory access at >=4 varlen segments under real-workload memory layouts (b26/b29, GeForce Blackwell) Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

Dao-AILab/flash-attention#2860 · 1 comment · Opened Sep 4, 2026
FA4 SM103 backward preprocess predicate shape mismatch for padded head dimensions Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

Dao-AILab/flash-attention#2852 · 1 comment · Opened Sep 4, 2026
splitkv early-exit path ignores unpadded_lse, writing -inf to the wrong LSE slot Open

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

Dao-AILab/flash-attention#2842 · 1 comment · Opened Sep 2, 2026
[BUG] swiglu_quant_mod FP8-store epilogue: cute.copy src ptr alignment (64 bits) does not meet requirement (128 bits) Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

Dao-AILab/quack#196 · Opened Sep 2, 2026
Windows build fails with CUDA 13.4: missing /Zc:preprocessor, /std:c++17 pinned (plus no sm_121 gencode) Open

Dao-AILab/flash-attention#2832 · 7 comments · 1 assignee · Opened Sep 1, 2026
Using `torch.library.wrap_triton` introduces measurable CPU overhead in `rotary` Open

Difficulty 4/5 3-5 days Newbie friendliness 55/100

Dao-AILab/flash-attention#2818 · 1 comment · Opened Aug 27, 2026
The v1.6.1 wheel is uploaded under the v1.6.2 release tag. Is this expected? Open

Difficulty 2/5 1-3 hours Newbie friendliness 50/100

Dao-AILab/causal-conv1d#121 · Opened Aug 25, 2026
tests/test_flash_attn.py fails collection without einops Open

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

Dao-AILab/flash-attention#2811 · Opened Aug 20, 2026
SM120? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Dao-AILab/gram-newton-schulz#24 · Opened Aug 18, 2026
No prebuilt wheels for sm_121 (GB10 / DGX Spark) Open

Difficulty 3/5 1-2 days Newbie friendliness 67/100

Dao-AILab/causal-conv1d#119 · Opened Aug 17, 2026
[FA4] max_seqlen_q tensor causes an argument-annotation mismatch in varlen forward Open

Difficulty 3/5 1-2 days Newbie friendliness 76/100

Dao-AILab/flash-attention#2801 · 1 comment · Opened Aug 17, 2026
[ROCm][CK][gfx1201][Windows] FlashAttention backward is 4.4–7.4x slower than PyTorch SDPA on RX 9070 XT Open

Difficulty 4/5 3-5 days Newbie friendliness 38/100

Dao-AILab/flash-attention#2800 · Opened Aug 17, 2026
Flash Attention install with Triton backend replaces preinstalled Triton no matter what. Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

Dao-AILab/flash-attention#2799 · 2 comments · Opened Aug 16, 2026
Verify evals on Papers with Code Open

Difficulty 3/5 1-2 days Newbie friendliness 48/100

Dao-AILab/sonic-moe#71 · Opened Aug 16, 2026
FA4 backward does not return against cutlass-dsl 4.6.0 and 4.6.1 on B300 (sm_103) Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

Dao-AILab/flash-attention#2797 · Opened Aug 15, 2026
Varlen version of `flash_attn_with_kvcache` Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

Dao-AILab/flash-attention#2793 · 1 reaction · Opened Aug 12, 2026
mbarrier_arrive uses legacy positional arguments and fails with current CUTLASS DSL wheels Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

Dao-AILab/quack#188 · Opened Aug 12, 2026
AllGatherRunner assumes six cudaStreamGetCaptureInfo return values and fails with cuda-bindings 12.9 Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

Dao-AILab/quack#187 · Opened Aug 12, 2026
ops/triton/linear.py kernel_fwd computes its bounds mask and never passes it to the store Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

Dao-AILab/flash-attention#2789 · Opened Aug 11, 2026
raise error cuda12.6 pytorch 2.10, vllm0.18.0 when install flash-attn==2.8.1 Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Dao-AILab/flash-attention#2783 · Opened Aug 10, 2026
publish.yml: create trigger has no tag filter, so branch pushes fail the workflow and new matrix entries never build Open

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

Dao-AILab/flash-attention#2769 · 1 comment · Opened Aug 7, 2026
[CuTe, SM100] test_clc_fuzz fails after #2559 due to stale scheduler expectations Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

Dao-AILab/flash-attention#2765 · Opened Aug 7, 2026
[Cute, SM100] FA4 and FA4_Varlen fwd runtime difference reaches 27% Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

Dao-AILab/flash-attention#2764 · Opened Aug 7, 2026
[CuTe, SM100] Deadlock in varlen + block-sparse + SplitKV forward: packed `split_idx` is not unpacked on the block-sparse paths (regression from #2559) Open

Difficulty 4/5 3-5 days Newbie friendliness 68/100

Dao-AILab/flash-attention#2760 · Opened Aug 6, 2026
flash_attn_func crashes with cudaErrorInvalidValue (SMEM overflow) at head_dim=256, causal, bf16 on sm_86/sm_89 (SM80 tile heuristic not head-dim-aware) Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

Dao-AILab/flash-attention#2750 · Opened Jul 31, 2026
[FA4/B200] return_lse=True makes D64 forward about 19% faster Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

Dao-AILab/flash-attention#2729 · 1 comment · Opened Jul 25, 2026
Plan to enable block size = 128 for block-sparse attention on FA4? (currently forced to 256) Open

Difficulty 5/5 Over a week Newbie friendliness 38/100

Dao-AILab/flash-attention#2727 · 1 comment · Opened Jul 23, 2026
FlashAttention-2 on AMD Radeon RX 9060 XT / gfx1200 with ROCm 7.14 Open

Difficulty 4/5 3-5 days Newbie friendliness 42/100

Dao-AILab/flash-attention#2725 · 1 comment · Opened Jul 23, 2026
[RFC] FlyDSL RMSNorm backend for ROCm Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

Dao-AILab/quack#178 · 1 comment · Opened Jul 22, 2026
Why do we need proxy fence in FA4 BWD postprocess kernel? Open

Dao-AILab/flash-attention#2718 · 2 comments · 1 reaction · 1 assignee · Opened Jul 19, 2026
Is there a plan for MXFP8 support? Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

Dao-AILab/flash-attention#2707 · 2 comments · Opened Jul 14, 2026
End-to-end DeepEP v2 non-expand + SonicMoE integration Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

Dao-AILab/sonic-moe#68 · 2 comments · Opened Jul 8, 2026
[Request]: Norm fwd: bucket `N` in compile cache key to reduce recompilations Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

Dao-AILab/quack#166 · Opened Jul 2, 2026
Investigate memory pressure around quack autotuner (OOM hangs) Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

Dao-AILab/gram-newton-schulz#23 · 1 comment · Opened Jul 2, 2026
Make dropout-disabled FA2 ABI stable with libtorch + cpython Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

Dao-AILab/flash-attention#2691 · 2 comments · Opened Jul 1, 2026
[Cute,Bwd,Sm100] Sparse MLA backward: two findings about dPsum stride divisibility and redundant elect Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

Dao-AILab/flash-attention#2677 · 9 comments · 1 reaction · Opened Jun 26, 2026
_flash_attn_fwd/_flash_attn_bwd: DSLRuntimeError — window_size_left/right expects Int32 but gets plain Python int Open

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

Dao-AILab/flash-attention#2653 · 1 comment · Opened Jun 13, 2026
atomic_add_fp32: TypeError on nvvm.atomicrmw(res=T.f32(), ...) — 'res' kwarg removed in cutlass-dsl 4.5.2 Open

Difficulty 1/5 Under an hour Newbie friendliness 78/100

Dao-AILab/flash-attention#2652 · 1 comment · Opened Jun 13, 2026
FlashAttentionBackwardSm80: compute_softmax_scale_log2 overwrites softmax_scale with None → DSLRuntimeError Open

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

Dao-AILab/flash-attention#2651 · 1 comment · Opened Jun 13, 2026
_flash_attn_bwd: UnboundLocalError for dQ_single_wg on SM_12x (arch//10 == 12) Open

Difficulty 1/5 Under an hour Newbie friendliness 84/100

Dao-AILab/flash-attention#2650 · 1 comment · Opened Jun 13, 2026
Can flash-attn 2.8.3 be built from source with PyTorch 2.11.0 + CUDA 13.0 on A800 (sm80)? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

Dao-AILab/flash-attention#2638 · Opened Jun 10, 2026
Performance Optimization: Fusing Pre-Attention Normalization to bridge the unfused GatedMLP / Layer bounds Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

Dao-AILab/flash-attention#2632 · Opened Jun 8, 2026
[Quick Question] in flash-attn 2.5.2, if I disable causal attn, does the attention_mask flash-attn 2 use remain the same or is it changed internally? Open

Difficulty 3/5 1-2 days Newbie friendliness 30/100

Dao-AILab/flash-attention#2631 · Opened Jun 6, 2026
Main is breaking Flash Attention 2.8.4 for RTX 4000 series GPUs - 2.8.3 working - RTX 3000 and 5000 series working Open

Difficulty 4/5 3-5 days Newbie friendliness 28/100

Dao-AILab/flash-attention#2619 · Opened Jun 2, 2026
Long-context attention on Apple Silicon: chunking-without-fusion fallback? Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

Dao-AILab/flash-attention#2603 · Opened May 29, 2026

Showing the newest 100

This page lists what was indexed most recently. Advanced filter has the whole inventory, narrowed by language, difficulty and how long a task takes.

Open advanced filter

Dao-AILab

Showing the newest 100

Get new issues in your inbox