Materialized encodings not compiling without flags.
- Dominant language
- C++
- Stars
- 3.9k
- Forks
- 1k
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 47
Description
I'm currently having trouble compiling extracted executables from a module with materialized encodings and it _may_ be related to global flags/non-hermetic configuration. My encodings look like:
```mlir
#encoding = #iree_encoding.layout<[#iree_gpu.gpu_encoding_resolver]>
#encoding1 = #iree_encoding.layout<[#iree_gpu.gpu_encoding_resolver]>
```
(produced by the MaterializeEncodingsPass)
In the context of offline parameter encoding we need to ensure all information required to lower the encodings through codegen is hermetically contained on the encoding attributes. The compiler invocation for the parameter encoding module will have a different device/executable target configuration and may be launched programmatically where we can't pass any original flags.
[Here](https://gist.github.com/benvanik/dd522b535632a19fceb04b823f2e51dd) is a module extracted from another which was originally compiled with:
```
--iree-hal-target-device=hip[0] \
--iree-hip-target=gfx1100 \
--iree-opt-data-tiling=false \
--iree-dispatch-creation-data-tiling=true \
--iree-hip-encoding-layout-resolver=data-tiling \
--iree-llvmgpu-test-combine-layout-transformation \
--iree-opt-level=O3 \
--iree-dispatch-creation-propagate-collapse-across-expands=true \
--iree-hal-indirect-command-buffers=true \
--iree-stream-resource-memory-model=discrete \
--iree-hal-memoization=true
```
and I'm trying to compile it without the `--iree-hip-encoding-layout-resolver=data-tiling` flag and failing in `MaterializeDeviceEncodingPass`:
```mlir
/home/ben/src/iree-tmp/encode_test.mlir:500:9: error: failed to legalize unresolved materialization from ('tensor<4x16x4x16x16xf16>') to ('tensor<256x256xf16>') that remained live after conversion
iree_tensor_ext.dispatch.tensor.store %3, %1, offsets = [0, 0], sizes = [256, 256], strides = [1, 1] : tensor<256x256xf16, #encoding1> -> !iree_tensor_ext.dispatch.tensor>
^
/home/ben/src/iree-tmp/encode_test.mlir:500:9: note: see current operation: %13 = "builtin.unrealized_conversion_cast"(%12) : (tensor<4x16x4x16x16xf16>) -> tensor<256x256xf16>
/home/ben/src/iree-tmp/encode_test.mlir:500:9: note: see existing live user here: iree_tensor_ext.dispatch.tensor.store %8, %4, offsets = [0, 0], sizes = [256, 256], strides = [1, 1] : tensor<256x256xf16> -> !iree_tensor_ext.dispatch.tensor>
/home/ben/src/iree-tmp/encode_test.mlir:494:7: error: 'func.func' op materialization failed
func.func @_encoding_4_encode_256x256xf16_to_256x256xf16(%arg0: !stream.binding, %arg1: !stream.binding) {
^
/home/ben/src/iree-tmp/encode_test.mlir:494:7: note: see current operation:
"func.func"() <{function_type = () -> (), sym_name = "_encoding_4_encode_256x256xf16_to_256x256xf16"}> ({
%0 = "arith.constant"() <{value = 0 : index}> : () -> index
%1 = "hal.interface.constant.load"() <{layout = #hal.pipeline.layout, #hal.pipeline.binding], flags = Indirect>, ordinal = 0 : index}> : () -> i32
%2 = "arith.index_castui"(%1) : (i32) -> index
%3 = "util.assume.int"(%2) <{assumptions = [[#util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption]]}> : (index) -> index
%4 = "hal.interface.binding.subspan"(%0) <{alignment = 64 : index, binding = 0 : index, descriptor_flags = 3 : i32, layout = #hal.pipeline.layout, #hal.pipeline.binding], flags = Indirect>, operandSegmentSizes = array}> : (index) -> !iree_tensor_ext.dispatch.tensor>
%5 = "hal.interface.binding.subspan"(%3) <{alignment = 64 : index, binding = 1 : index, descriptor_flags = 2 : i32, layout = #hal.pipeline.layout, #hal.pipeline.binding], flags = Indirect>, operandSegmentSizes = array}> : (index) -> !iree_tensor_ext.dispatch.tensor>
%6 = "iree_tensor_ext.dispatch.tensor.load"(%4) <{operandSegmentSizes = array, static_offsets = array, static_sizes = array, static_strides = array}> : (!iree_tensor_ext.dispatch.tensor>) -> tensor<256x256xf16>
%7 = "arith.constant"() <{value = 0.000000e+00 : f16}> : () -> f16
%8 = "tensor.empty"() : () -> tensor<4x16x64x16xf16>
%9 = "linalg.pack"(%6, %8, %7) <{inner_dims_pos = array, operandSegmentSizes = array, outer_dims_perm = array, static_inner_tiles = array}> : (tensor<256x256xf16>, tensor<4x16x64x16xf16>, f16) -> tensor<4x16x64x16xf16>
%10 = "tensor.expand_shape"(%9) <{reassociation = [[0], [1], [2, 3], [4]], static_output_shape = array}> : (tensor<4x16x64x16xf16>) -> tensor<4x16x4x16x16xf16>
%11 = "tensor.empty"() : () -> tensor<4x16x4x16x16xf16>
%12 = "linalg.transpose"(%10, %11) <{permutation = array}> ({
^bb0(%arg0: f16, %arg1: f16):
"linalg.yield"(%arg0) : (f16) -> ()
}) : (tensor<4x16x4x16x16xf16>, tensor<4x16x4x16x16xf16>) -> tensor<4x16x4x16x16xf16>
%13 = "builtin.unrealized_conversion_cast"(%12) : (tensor<4x16x4x16x16xf16>) -> tensor<256x256xf16>
"iree_tensor_ext.dispatch.tensor.store"(%13, %5) <{operandSegmentSizes = array, static_offsets = array, static_sizes = array, static_strides = array}> : (tensor<256x256xf16>, !iree_tensor_ext.dispatch.tensor>) -> ()
"func.return"() : () -> ()
}) : () -> ()
```
In the future I'll be removing all those flags - the module below _should_ compile with just:
```mlir
--iree-hal-target-device=hip \
--iree-hip-target=gfx1100
```
and then the next step is that it should compile with:
```mlir
--iree-hal-target-device=local \
--iree-hal-local-target-device-backends=llvm-cpu
```
Contributor guide
Assessment
This issue has not been assessed yet.