deepsweet / deepsweet/mlx-eval

Enhanced quantization (oQe)

Open
#6 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
26
Forks
0
PR merge metrics
No merged PRs in 30d

Description

This might be a really silly question, however I notice that you haven't included any of the Enhanced quantization (oQe) levels.
My understanding from [https://github.com/jundot/omlx/blob/main/docs/oQ_Quantization.md](https://github.com/jundot/omlx/blob/main/docs/oQ_Quantization.md) is that these enhanced quants add more precision, especially to the lower quants.
Do you plan to include these in the future, or have I misunderstood what oQe is?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the linked docs/oQ_Quantization.md to understand the oQe levels and how they differ from the quantizations currently evaluated. Then inspect mlx-eval's quantization-related entry points and document the required scope; this issue is done only when the supported levels and evaluation expectations are agreed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.