deepseek-ai / deepseek-ai/FlashMLA
Are the test performance results of tests/test_flash_mla.py accurate?
Open
- Dominant language
- C++
- Stars
- 12.9k
- Forks
- 1.2k
- Avg merge
- 4h 20m
- Merged PRs (30d)
- 2
Description
def flash_mla():
torch.cuda.synchronize()
tile_scheduler_metadata, num_splits = get_mla_metadata(cache_seqlens, s_q * h_q // h_kv, h_kv)
I added a `sync()`, and found that the performance was much worse. With `sync()`, it took 360us, while without it, it only took 50us.
Why does it feel like the cost time is a CPU's time? (The kernel submits asynchronously and hasn't finished executing yet.)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.