iree-org / iree-org/iree

[codegen][gpu] Performance investigation of default convolution filter layout

Open
#20,105 0 comments 0 reactions 0 assignees View on GitHub
enhancement ➕
Dominant language
C++
Stars
3.9k
Forks
1k
Avg merge
4d 16h
Merged PRs (30d)
47

Description

### Request description

This is a medium/low priority follow-up of https://github.com/iree-org/iree/issues/19701.

By the end of the ticket, we decide to use `FHWC` layout for filter by default in the preprocessing pipeline https://github.com/iree-org/iree/pull/19974. This is due to:
- For `FHWC` layout, the gemmK dimensions `HWC` will be collapsed as a single reduction dimension which is cleaner compared with `HWFC` layout
- `FHWC` is a more common choice among different framework and libraries
- `FHWC` and `HWFC`, in the limited tuning benchmarking, delivers around similar performance

However, the performance evaluation is not carried in a comprehensive manner and we may be able to adopt a better default layout in different scenarios. This ticket is used to track and further determine the right default layout for convolution in the preprocessing pipelines.

To finish this, we'd like a comprehensive study of the layout impact on convolution tuned performance and document our studies, alter the implementation of `iree-preprocessing-convert-conv-filter-to-channels-last` when necessary.

### What component(s) does this issue relate to?

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.