Quantized distillgpt2 accuracy issue when quant params are inputs
Open
@shivadbhavsar is already working on this.
Since Nov 11, 2024.
bug
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
Issue originating from pytorch-quantized model where the scales and zero points are passed as inputs rather than being embedded in the model as literals.
MXR for small problematic block can be found in nas at: /migraphx/models/torch_exports/distilgpt2_block_torch.mxr
to reproduce use:
migraphx-driver verify distilgpt2_block_torch.mxr --fill1 arg0_1 --fill0 arg4_1 --fill0 arg10_1 --fill0 arg12_1 --fill0 arg14_1
This verification only fails after #3362
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.