iree-org / iree-org/fusilli

[Dispatch creation] Grouped forward convs include a `slow_memcpy` dispatch

Open
#7 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
25
Forks
16
PR merge metrics
No merged PRs in 30d

Description

Looking at the forward group conv commands, e.g., `--iter 1 conv --bf16 -F 1 -n 1 -c 64 -H 48 -W 32 -k 64 -y 3 -x 3 -p 1 -q 1 -u 1 -v 1 -l 1 -j 1 -g 4 --in_layout NHWC --fil_layout NHWC --out_layout NHWC --spatial_dim 2`, we get two dispatches:

- `main$async_dispatch_0_slow_memcpy` and
- `main$async_dispatch_1_conv_4x16x48x32x16x3x3_bf16xbf16xf32` (for example)
We have `slow_memcpy` before the AnnotateDispatchesPasses:
```
builtin.module {
func.func @main$async_dispatch_0(%arg0: !iree_tensor_ext.dispatch.tensor>, %arg1: !iree_tensor_ext.dispatch.tensor>) {
%0 = iree_tensor_ext.dispatch.tensor.load %arg0, offsets = [0, 0, 0], sizes = [48, 32, 64], strides = [1, 1, 1] : !iree_tensor_ext.dispatch.tensor> -> tensor<48x32x64xbf16>
iree_tensor_ext.dispatch.tensor.store %0, %arg1, offsets = [1, 1, 0], sizes = [48, 32, 64], strides = [1, 1, 1] : tensor<48x32x64xbf16> -> !iree_tensor_ext.dispatch.tensor>
return
}
}
```
and we can see that this is an insert slice operation from after FormDispatchRegionsPass ([MLIR](https://gist.github.com/raayandhar/80c8b2514726df9777e5b0b1cbae6baa)).

We should probably not have this be an individual dispatch. I was able to reproduce this across groups 2 through 6. For reproducibility, I have the [iree-compile-command.txt](https://gist.github.com/raayandhar/d57cd7510a87bc317365b3ae7029e788) and [iree-compile-input.mlir](https://gist.github.com/raayandhar/c29ee06040056b691c0ea86adacf07ec).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.