deepseek-ai / deepseek-ai/FlashMLA

Are the test performance results of tests/test_flash_mla.py accurate?

Open
#59 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
12.9k
Forks
1.2k
Avg merge
4h 20m
Merged PRs (30d)
2

Description

    def flash_mla():

torch.cuda.synchronize()
tile_scheduler_metadata, num_splits = get_mla_metadata(cache_seqlens, s_q * h_q // h_kv, h_kv)

I added a `sync()`, and found that the performance was much worse. With `sync()`, it took 360us, while without it, it only took 50us.

Why does it feel like the cost time is a CPU's time? (The kernel submits asynchronously and hasn't finished executing yet.)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.