qwen1.5 not giving attention kernels for int8
Open
@aarushjain29 is already working on this.
Since Jun 23, 2025.
- Dominant language
- C++
- Stars
- 333
- Forks
- 150
- Avg merge
- 4d 19h
- Merged PRs (30d)
- 54
Description
#qwen1_5_7b_int8
-t f32 -out_datatype f32 -transA false -transB false -g 1 -m 256 -n 22016 -k 4096
-t f32 -out_datatype f32 -transA false -transB true -g 32 -m 256 -n 256 -k 128
-t f32 -out_datatype f32 -transA false -transB false -g 32 -m 256 -n 128 -k 256
-t f32 -out_datatype f32 -transA false -transB false -g 1 -m 256 -n 12288 -k 4096
-t f32 -out_datatype f32 -transA false -transB false -g 1 -m 256 -n 4096 -k 11008
-t f32 -out_datatype f32 -transA false -transB false -g 1 -m 256 -n 4096 -k 4096
-t f32 -out_datatype f32 -transA false -transB false -g 1 -m 256 -n 151936 -k 4096
-t i8 -out_datatype i32 -transA false -transB true -g 32 -m 256 -n 256 -k 128
-t i8 -out_datatype i32 -transA false -transB false -g 1 -m 256 -n 4096 -k 11008
-t i8 -out_datatype i32 -transA false -transB false -g 1 -m 256 -n 12288 -k 4096
-t i8 -out_datatype i32 -transA false -transB false -g 1 -m 256 -n 4096 -k 4096
-t i8 -out_datatype i32 -transA false -transB false -g 1 -m 256 -n 22016 -k 4096
-t i8 -out_datatype i32 -transA false -transB false -g 32 -m 256 -n 128 -k 256
-t i8 -out_datatype i32 -transA false -transB false -g 1 -m 256 -n 151936 -k 4096Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.