AI-Hypercomputer / AI-Hypercomputer/accelerator-microbenchmarks
[Question] I'm confused about what the "_xd" suffix means in the all_to_all benchmark configs
- Vorherrschende Sprache
- Python
- Sterne
- 24
- Forks
- 42
- Ø Merge
- 3 Std. 36 Min.
- Gemergte PRs (30 T.)
- 2
Beschreibung
When reading the accelerator-microbenchmarks code, I'm very confused about one part. Below is the benchmark test configuration for all_to_all communication:
all_to_all_3d:
- mesh_shape: "4x4x8"
sharding_strategy: "4x4x8"
op_dimension: 3
all_to_all_2d:
- mesh_shape: "4x32"
sharding_strategy: "1x32"
op_dimension: 2
all_to_all_1d:
mesh_shape: "4x4x8"
sharding_strategy: "1x1x4"
op_dimension: 1
I originally thought that "_xd" describes data communicating across x dimensions, but it seems that both 1d and 2d only need communication in one dimension? Is it determined by op_dimension for how many dimensions? The all_to_all_benchmark function in the code also doesn't seem to use the op_dimension parameter.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.