All2all performance comparison between Triton-distributed and DeepEP

---

# Performance Comparison: `triton-distributed` vs `DeepEP` on Single-Node H20 (8 GPUs)

We benchmarked the performance of two implementations on a single-node H20 system with 8 GPUs:

- **DeepEP**: Using the provided test script — `tests/test_low_latency.py`
- **triton-distributed**: Using `/python/triton_dist/test/nvidia/test_all_to_all.py`, but **only measuring the kernel execution time** (excluding the gather overhead before communication).

The tests were split into two phases: **dispatch** and **combine**.

---

## Test Code Snippet for `triton-distributed`

```python
from triton_dist.utils import perf_func
from functools import partial

def fast_all_to_all_for_time_only_kernel(ctx, send_split_cumsum, with_scale):
grid = (ctx.WORLD_SIZE, )
all_to_all_kernel[grid](
data_src=ctx.send_buf,
data_dst=ctx.recv_buf,
splits_src=ctx.split_send_buf,
splits_dst=ctx.split_recv_buf,
signal=ctx.signal_buf,

splits_cumsum=send_split_cumsum,

scale_src=ctx.scale_send_buf,
scale_dst=ctx.scale_recv_buf,
rank=ctx.rank,
call_count=ctx.call_count,
WITH_SCALE=with_scale,

WORLD_SIZE=ctx.WORLD_SIZE,
HIDDEN=ctx.hidden,
MAX_M=ctx.max_m,
EXPERTS_PER_RANK=ctx.experts_per_rank,
NUM_TOT_EXPERTS=ctx.num_tot_experts,
ELEMENT_SIZE=ctx.ele_size,
SCALE_ELEMENT_SIZE=ctx.scale_ele_size,
)
return ctx

def fast_all_to_all(...):
...
ctx, all_to_all_time = perf_func(
partial(fast_all_to_all_for_time_only_kernel, ctx, send_split_cumsum, with_scale),
iters=100,
warmup_iters=20
)
```

---

## Benchmark Configuration

- `N = 7168`
- `G = 256`
- `topk = 8`
- Data type: `bfloat16`

---

## Results (Latency in μs)

| M | DeepEP (dispatch / combine) | triton-distributed (dispatch / combine) |
|-----|-----------------------------|------------------------------------------|
| 2 | 15.41 / 12.13 | 14 / 11 |
| 4 | 15.56 / 12.85 | 13 / 15 |
| 8 | 16.94 / 13.73 | 25 / 20 |
| 16 | 16.30 / 17.35 | 26 / 28 |
| 32 | 20.45 / 24.24 | 47 / 50 |
| 64 | 27.76 / 41.32 | 80 / 88 |
| 128 | 37.08 / 71.75 | 145 / 170 |
| 256 | 64.40 / 140.42 | 341 / 398 |
| 512 | 120.83 / 271.19 | 1180 / 1443 |

> ⚠️ Note: `triton-distributed` results **exclude gather overhead**, while `DeepEP` includes full end-to-end cost.

---

## Questions

1. Do these results align with the expected performance advantage of `triton-distributed` over `DeepEP` on small M?
— We observe that for small `M`, `triton-distributed` is competitive or slightly better. However, it does not achieve a more efficient communication acceleration effect when the overhead of data gather is also taken into account.

2. Are there any upcoming **more efficient open-source communication implementations** planned for `triton-distributed` that could further optimize all-to-all performance?
— Programs (besides test_ep_a2a.py for higher M)
---

Thank you for your time and any insights you can provide!

---

Start by reviewing tests/test_low_latency.py and /python/triton_dist/test/nvidia/test_all_to_all.py, then compare their measurement scopes and the reported configurations. Check test_ep_a2a.py for the higher-M program mentioned in the issue. This issue is complete only when the performance questions are answered or a specific optimization task is agreed.

Written by the indexing model from the issue text.