iree-org / iree-org/iree

Materialized encodings not compiling without flags.

Open
#22,073 1 comment 0 reactions 1 assignee Claimed by @hanhanW View on GitHub
codegen
Dominant language
C++
Stars
3.9k
Forks
1k
Avg merge
4d 16h
Merged PRs (30d)
47

Description

I'm currently having trouble compiling extracted executables from a module with materialized encodings and it _may_ be related to global flags/non-hermetic configuration. My encodings look like:
```mlir
#encoding = #iree_encoding.layout<[#iree_gpu.gpu_encoding_resolver]>
#encoding1 = #iree_encoding.layout<[#iree_gpu.gpu_encoding_resolver]>
```
(produced by the MaterializeEncodingsPass)

In the context of offline parameter encoding we need to ensure all information required to lower the encodings through codegen is hermetically contained on the encoding attributes. The compiler invocation for the parameter encoding module will have a different device/executable target configuration and may be launched programmatically where we can't pass any original flags.

[Here](https://gist.github.com/benvanik/dd522b535632a19fceb04b823f2e51dd) is a module extracted from another which was originally compiled with:
```
--iree-hal-target-device=hip[0] \
--iree-hip-target=gfx1100 \
--iree-opt-data-tiling=false \
--iree-dispatch-creation-data-tiling=true \
--iree-hip-encoding-layout-resolver=data-tiling \
--iree-llvmgpu-test-combine-layout-transformation \
--iree-opt-level=O3 \
--iree-dispatch-creation-propagate-collapse-across-expands=true \
--iree-hal-indirect-command-buffers=true \
--iree-stream-resource-memory-model=discrete \
--iree-hal-memoization=true
```

and I'm trying to compile it without the `--iree-hip-encoding-layout-resolver=data-tiling` flag and failing in `MaterializeDeviceEncodingPass`:
```mlir
/home/ben/src/iree-tmp/encode_test.mlir:500:9: error: failed to legalize unresolved materialization from ('tensor<4x16x4x16x16xf16>') to ('tensor<256x256xf16>') that remained live after conversion
iree_tensor_ext.dispatch.tensor.store %3, %1, offsets = [0, 0], sizes = [256, 256], strides = [1, 1] : tensor<256x256xf16, #encoding1> -> !iree_tensor_ext.dispatch.tensor>
^
/home/ben/src/iree-tmp/encode_test.mlir:500:9: note: see current operation: %13 = "builtin.unrealized_conversion_cast"(%12) : (tensor<4x16x4x16x16xf16>) -> tensor<256x256xf16>
/home/ben/src/iree-tmp/encode_test.mlir:500:9: note: see existing live user here: iree_tensor_ext.dispatch.tensor.store %8, %4, offsets = [0, 0], sizes = [256, 256], strides = [1, 1] : tensor<256x256xf16> -> !iree_tensor_ext.dispatch.tensor>
/home/ben/src/iree-tmp/encode_test.mlir:494:7: error: 'func.func' op materialization failed
func.func @_encoding_4_encode_256x256xf16_to_256x256xf16(%arg0: !stream.binding, %arg1: !stream.binding) {
^
/home/ben/src/iree-tmp/encode_test.mlir:494:7: note: see current operation:
"func.func"() <{function_type = () -> (), sym_name = "_encoding_4_encode_256x256xf16_to_256x256xf16"}> ({
%0 = "arith.constant"() <{value = 0 : index}> : () -> index
%1 = "hal.interface.constant.load"() <{layout = #hal.pipeline.layout, #hal.pipeline.binding], flags = Indirect>, ordinal = 0 : index}> : () -> i32
%2 = "arith.index_castui"(%1) : (i32) -> index
%3 = "util.assume.int"(%2) <{assumptions = [[#util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption, #util.int.assumption]]}> : (index) -> index
%4 = "hal.interface.binding.subspan"(%0) <{alignment = 64 : index, binding = 0 : index, descriptor_flags = 3 : i32, layout = #hal.pipeline.layout, #hal.pipeline.binding], flags = Indirect>, operandSegmentSizes = array}> : (index) -> !iree_tensor_ext.dispatch.tensor>
%5 = "hal.interface.binding.subspan"(%3) <{alignment = 64 : index, binding = 1 : index, descriptor_flags = 2 : i32, layout = #hal.pipeline.layout, #hal.pipeline.binding], flags = Indirect>, operandSegmentSizes = array}> : (index) -> !iree_tensor_ext.dispatch.tensor>
%6 = "iree_tensor_ext.dispatch.tensor.load"(%4) <{operandSegmentSizes = array, static_offsets = array, static_sizes = array, static_strides = array}> : (!iree_tensor_ext.dispatch.tensor>) -> tensor<256x256xf16>
%7 = "arith.constant"() <{value = 0.000000e+00 : f16}> : () -> f16
%8 = "tensor.empty"() : () -> tensor<4x16x64x16xf16>
%9 = "linalg.pack"(%6, %8, %7) <{inner_dims_pos = array, operandSegmentSizes = array, outer_dims_perm = array, static_inner_tiles = array}> : (tensor<256x256xf16>, tensor<4x16x64x16xf16>, f16) -> tensor<4x16x64x16xf16>
%10 = "tensor.expand_shape"(%9) <{reassociation = [[0], [1], [2, 3], [4]], static_output_shape = array}> : (tensor<4x16x64x16xf16>) -> tensor<4x16x4x16x16xf16>
%11 = "tensor.empty"() : () -> tensor<4x16x4x16x16xf16>
%12 = "linalg.transpose"(%10, %11) <{permutation = array}> ({
^bb0(%arg0: f16, %arg1: f16):
"linalg.yield"(%arg0) : (f16) -> ()
}) : (tensor<4x16x4x16x16xf16>, tensor<4x16x4x16x16xf16>) -> tensor<4x16x4x16x16xf16>
%13 = "builtin.unrealized_conversion_cast"(%12) : (tensor<4x16x4x16x16xf16>) -> tensor<256x256xf16>
"iree_tensor_ext.dispatch.tensor.store"(%13, %5) <{operandSegmentSizes = array, static_offsets = array, static_sizes = array, static_strides = array}> : (tensor<256x256xf16>, !iree_tensor_ext.dispatch.tensor>) -> ()
"func.return"() : () -> ()
}) : () -> ()
```

In the future I'll be removing all those flags - the module below _should_ compile with just:
```mlir
--iree-hal-target-device=hip \
--iree-hip-target=gfx1100
```
and then the next step is that it should compile with:
```mlir
--iree-hal-target-device=local \
--iree-hal-local-target-device-backends=llvm-cpu
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.