intel / intel/auto-round

[vllm-omni]: Omni Quant Support

Open
#1,507 2 comments 0 reactions 1 assignee Claimed by @thuang6 View on GitHub
enhancement high priority
Dominant language
Python
Stars
1.6k
Forks
175
Avg merge
1d 18h
Merged PRs (30d)
99

Description

### Feature Description

This epic is created to track Omni model (vllm-omni targeted) quantization support status.
main dtypes includes: INT4, MXFP8, MXFP4 is future target

### Motivation and Use Case

### omni model enable status:

| model | architecture | owner| INT4 | MXFP8 |
|---------|---------------|--------|-------|----------|
| Qwen3-omni | Qwen3OmniMoeForConditionalGeneration | Liang | ok | |
| Qwen2.5-omni | Qwen2_5OmniForConditionalGeneration | Liang | ok | |
| GLM-Image | GlmImageForConditionalGeneration GlmImageTransformer2DModel| Liang | ok| |
| BAGEL-7B-MoT | BagelForConditionalGeneration | Liang | ok | |
| Ovis-Image | OvisImageTransformer2DModel| Liang | ok | |
| HunyuanImage-3.0 | HunyuanImage3ForCausalMM | Chang | ok | |
| OmniGen2 | - | Chang | WIP | |
| Z-Image | ZImageTransformer2DModel| Xin | ok | |
| Next-Step | NextStep | Xin | WIP ([Feature request in vllm-omni](https://github.com/vllm-project/vllm-omni/issues/1815)) | |
| LongCat-Image | LongCatImagePipeline | Xin | WIP | |
| MiMo-Audio-7B-Instruct | MiMoAudioForCausalLM | Weiwei | ok| |
| Qwen3-TTS-12Hz-1.7B-CustomVoice | Qwen3TTSForConditionalGeneration | Weiwei | ok| |
| stable-audio-open | - | Weiwei | ok| |
| FLUX.1-dev | FluxTransformer2DModel | Mengni | ok | |
| SD | - | Mengni | - | |
| Wan | WanTransformer3DModel | Mengni/Liang | ok | |
| QwenImage | - | Mengni | - | |

### vllm-omni Support Status
https://github.com/vllm-project/vllm-omni/issues/1325
| HF model id | Type | status | owner | PR |
| ------------------------------------------------------------ | ---------------------- | ------ | ----- | ------------------------------------------------------------ |
| [black-forest-labs/FLUX.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) | text→image | Merged | Yi | [#1777](https://github.com/vllm-project/vllm-omni/pull/1777) |
| [Qwen/Qwen3-Omni-30B-A3B-Instruct](https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct) | text+audio→text+audio | Merged| Liang | [#2670](https://github.com/vllm-project/vllm-omni/pull/2670#top) |
| [Qwen/Qwen2.5-Omni-7B](https://huggingface.co/Qwen/Qwen2.5-Omni-7B) | text+audio→text+audio | Merged| Liang | [#2670](https://github.com/vllm-project/vllm-omni/pull/2670#top) |
| [Qwen/Qwen-Image](https://huggingface.co/Qwen/Qwen-Image) | text→image | Merged | Yi | [#3588](https://github.com/vllm-project/vllm-omni/pull/3588) |
| [Wan-AI/Wan2.2-T2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-T2V-A14B-Diffusers) | text→video | Merged | Liang | [#3353](https://github.com/vllm-project/vllm-omni/pull/3353) |
| [Wan-AI/Wan2.2-I2V-A14B-Diffusers](https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B-Diffusers) | image→video | Merged | Liang | [#3353](https://github.com/vllm-project/vllm-omni/pull/3353) |
| [Wan-AI/Wan2.2-S2V-14B](https://huggingface.co/Wan-AI/Wan2.2-S2V-14B) | audio+image+text→video | WIP | Liang | |
| [zai-org/GLM-Image](https://huggingface.co/zai-org/GLM-Image) | text→image | Merged| Liang | [#3059](https://github.com/vllm-project/vllm-omni/pull/3059) |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.