int8 quantization refinement for convolution

Open
#1,170 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
cpp
Domain
machine-learning, performance

Research direction

Start by reading PR #881, which removed the int8x4 packing for rocblas GEMM, then locate the convolution path that still performs this packing. Compare its rocblas call requirements for MI100 and verify that convolution int8 quantization no longer incurs the packing overhead.

Written by the indexing model from the issue text.

Description

Current implementation of the int8 convolution needs to pack input data to int8x4 data format, which is similar to the int8x4 data format requirement as the rocblas api. This packing is very slow, which make the int8 quantization is slower than fp16. From rocblas version 2.38 (maybe rocm4.0, software) and mi100 (hardware), this int8x4 data format requirement is not needed anymore.

We made the changes to remove the packing procedure for calling to rocblas (PR#881), so int8 gemm is supposed to running better on mi100 than before. We also need to make the change related to the convolution.

This issue is to track that changes.

Dominant language
C++
Stars
333
Forks
150
Avg merge
4d 19h
Merged PRs (30d)
54

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ROCm/AMDMIGraphX

All issues in ROCm/AMDMIGraphX

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.