[QST] Quantized conv with s8 output and s32 bias
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 10.5k
- Forks
- 2.1k
- Avg merge
- 3d 11h
- Merged PRs (30d)
- 7
Description
When implementing a quantized GEMM/convolution with INT8 activations and weights, it's common to also have the bias as INT32. The usual trick for adding a bias seems to be initializing the C matrix to the bias with a stride of 0. This approach would require ElementC to be declared as INT32, yet I also want the output of the convolution to be INT8; it seems ElementC is also implicitly the output data type as far as I can tell.
It's not clear how to achieve what I want with 2.x APIs/epilogues. Do I need to use EVT to accomplish this?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the CUTLASS 2.x GEMM/convolution APIs and epilogues referenced in the issue, focusing on how ElementC determines the accumulator input and output types. Investigate whether EVT supports an INT32 bias with an INT8 result, and document the supported approach or limitation as the completion criterion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- hpc, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100