alibaba / alibaba/EfficientAI

Question about MASQuant

Open
#4 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
51
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Hi authors, thanks for sharing the excellent MASQuant work!
I have a quick question about the **W4A4 setting** in Table 7. I noticed that **no W4A4 accuracy metrics (MMMU/OCR/VQA/WER)** are reported in Table 1 and Table 2, while Table 7 only shows the inference speed/memory under W4A4.
Could you clarify whether the W4A4 configuration in Table 7 is **only for inference efficiency evaluation** and not fully quantized with reported accuracy? If W4A4 quantization is fully implemented, could you please provide the corresponding accuracy results?
Thanks a lot!

Contributor guide

No contributing guide indexed for this repository

Research direction

Review the W4A4 entries in Tables 1, 2, and 7 of the MASQuant work. Confirm whether Table 7 reports only inference speed and memory, or whether corresponding MMMU, OCR, VQA, and WER accuracy results exist; done means providing a clear clarification or adding the missing results.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.