Horizontal Fusion

Horizontal fusion

@521 = gpu::code_object[code_object=9352,symbol_name=add_relu_kernel,global=12288,local=1024,](@513,@520,@519) -> float_type, {1, 384, 8, 8}, {24576, 64, 8, 1}
@526 = hip::hip_copy_literal[id=main:@literal:27] -> float_type, {384, 384, 3, 1}, {1152, 3, 1, 1}
@527 = hip::hip_copy_literal[id=main:@literal:28] -> float_type, {384, 384, 1, 3}, {1152, 3, 3, 1}
@534 = gpu::code_object[code_object=6136,symbol_name=mlir_convolution,global=6144,local=64,](@521,@526,@533) -> float_type, {1, 384, 8, 8}, {24576, 64, 8, 1}
@537 = load[offset=245760,end=344064](@1) -> float_type, {1, 384, 8, 8}, {24576, 64, 8, 1}
@538 = gpu::code_object[code_object=6712,symbol_name=mlir_convolution,global=6144,local=64,](@521,@527,@537) -> float_type, {1, 384, 8, 8}, {24576, 64, 8, 1}
@539 = load[offset=425984,end=524288](@1) -> float_type, {1, 384, 8, 8}, {24576, 64, 8, 1}

We can horizontally fuse these convolutions into a single convolution where we concat the weights. We currently dont do that because the filter sizes are different. Since this uses same padding mode we can just pad the weights to make them both 3x3 and then we can fuse them.