[Dispatch creation] Grouped forward convs include a `slow_memcpy` dispatch
- Dominant language
- C++
- Stars
- 25
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
Looking at the forward group conv commands, e.g., `--iter 1 conv --bf16 -F 1 -n 1 -c 64 -H 48 -W 32 -k 64 -y 3 -x 3 -p 1 -q 1 -u 1 -v 1 -l 1 -j 1 -g 4 --in_layout NHWC --fil_layout NHWC --out_layout NHWC --spatial_dim 2`, we get two dispatches:
- `main$async_dispatch_0_slow_memcpy` and
- `main$async_dispatch_1_conv_4x16x48x32x16x3x3_bf16xbf16xf32` (for example)
We have `slow_memcpy` before the AnnotateDispatchesPasses:
```
builtin.module {
func.func @main$async_dispatch_0(%arg0: !iree_tensor_ext.dispatch.tensor>, %arg1: !iree_tensor_ext.dispatch.tensor>) {
%0 = iree_tensor_ext.dispatch.tensor.load %arg0, offsets = [0, 0, 0], sizes = [48, 32, 64], strides = [1, 1, 1] : !iree_tensor_ext.dispatch.tensor> -> tensor<48x32x64xbf16>
iree_tensor_ext.dispatch.tensor.store %0, %arg1, offsets = [1, 1, 0], sizes = [48, 32, 64], strides = [1, 1, 1] : tensor<48x32x64xbf16> -> !iree_tensor_ext.dispatch.tensor>
return
}
}
```
and we can see that this is an insert slice operation from after FormDispatchRegionsPass ([MLIR](https://gist.github.com/raayandhar/80c8b2514726df9777e5b0b1cbae6baa)).
We should probably not have this be an individual dispatch. I was able to reproduce this across groups 2 through 6. For reproducibility, I have the [iree-compile-command.txt](https://gist.github.com/raayandhar/d57cd7510a87bc317365b3ae7029e788) and [iree-compile-input.mlir](https://gist.github.com/raayandhar/c29ee06040056b691c0ea86adacf07ec).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.