Identify and fix issues with `bnb-quantized*` quantization
- Dominant language
- Python
- Stars
- 18
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
There are issues with `bnb-quantized*` quantization. For example, in the test `kvcache/test_quantization.py -k test_quantization_error`,
we skip `blocks_over_heads = True, name = 'dense-bnb-quantized*`.
**To reproduce**
Remove the skip in the test and run it (on GPU).
**Expected behavior**
`bnb-quantized*`should either work or be removed.
Contributor guide
Research direction
Start with kvcache/test_quantization.py and remove the skip for blocks_over_heads = True, name = 'dense-bnb-quantized*'. Run test_quantization_error on a GPU to observe the failure and trace the affected quantization path. Done means the test passes for the quantized variant, or the variant is removed if it cannot work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100