A8W4 fuse moe kernel in testx branch issue
@coderfeli is already working on this.
Since Jul 4, 2025.
Assessment
This issue has not been assessed yet.
Description
I modified the inter_dim in https://github.com/ROCm/aiter/blob/d9c1a7390ec27b7bd948f9233f1e49a8fef82d54/op_tests/test_moe_2stage_int4.py#L397 from 4096 to 256, and got
RuntimeError: wrong! device_gemm with the specified compilation parameters does not support this GEMM problem
This inter_dim is needed when deploying deepseek on MI300x with TP8.
To solve this problem, I tried to change the PrefetchStages in https://github.com/ROCm/composable_kernel/blob/b20173a49471a9e22154654fdddec8fe088a4e27/include/ck/tensor_operation/gpu/block/blockwise_gemm_pipeline_xdlops_b_preshuffle_dequant_v1.hpp#L145 from 2 to 1, and rebuild aiter. After that the test_moe_2stage_int4.py unit test was able to run, but gave the wrong results.
- Dominant language
- Python
- Stars
- 565
- Forks
- 585
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 366
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from ROCm/aiter
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
[MLA][gfx942] Native qh64 fp8 persistent decode kernel GPU-faults at page_size=1 (block_size=1) Open
Difficulty 2/5 Half a day Newbie friendliness 72/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bancolombia/sentinel#23 ·
-
test md OpenCI
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
bug client
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100