AI-Hypercomputer / AI-Hypercomputer/accelerator-microbenchmarks

[Question] I'm confused about what the "_xd" suffix means in the all_to_all benchmark configs

未關閉
#72 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
24
分支
42
平均合併
3 小時 36 分鐘
30 天內合併 PR
2

描述

When reading the accelerator-microbenchmarks code, I'm very confused about one part. Below is the benchmark test configuration for all_to_all communication:

all_to_all_3d:
- mesh_shape: "4x4x8"
sharding_strategy: "4x4x8"
op_dimension: 3

all_to_all_2d:
- mesh_shape: "4x32"
sharding_strategy: "1x32"
op_dimension: 2

all_to_all_1d:
mesh_shape: "4x4x8"
sharding_strategy: "1x1x4"
op_dimension: 1

I originally thought that "_xd" describes data communicating across x dimensions, but it seems that both 1d and 2d only need communication in one dimension? Is it determined by op_dimension for how many dimensions? The all_to_all_benchmark function in the code also doesn't seem to use the op_dimension parameter.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。