NVIDIA / NVIDIA/TensorRT-Edge-LLM
visual_quantization: invalid choice: 'mxfp8'
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 563
- Forks
- 135
- Avg merge
- 14h 13m
- Merged PRs (30d)
- 1
Description
(trt_edgellm) lz@10-60-57-2:~/hf/models$ tensorrt-edgellm-quantize llm \
--model_dir /home/lz/hf/models/Qwen_Qwen3-VL-2B-Instruct \
--output_dir /home/lz/hf/models/q3vl_mxfp8 \
--quantization mxfp8 \
--lm_head_quantization mxfp8 \
--visual_quantization mxfp8
usage: tensorrt-edgellm-quantize llm [-h] --model_dir MODEL_DIR --output_dir OUTPUT_DIR
[--quantization {fp8,int4_awq,nvfp4,mxfp8,int8_sq}]
[--lm_head_quantization {fp8,int4_awq,nvfp4,mxfp8}]
[--visual_quantization {fp8}] [--audio_quantization {fp8}]
[--kv_cache_quantization {fp8}] [--dtype {fp16}] [--device DEVICE]
[--dataset DATASET] [--num_samples NUM_SAMPLES]
tensorrt-edgellm-quantize llm: error: argument --visual_quantization: invalid choice: 'mxfp8' (choose from 'fp8')
Why is MXFP8 quantization not supported for the visual encoder?
If I want to customize support for MXFP8 quantization for the visual encoder, what should I do?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the tensorrt-edgellm-quantize llm CLI entry point and inspect how the --visual_quantization choices are defined and handled. Reproduce the command shown with --visual_quantization mxfp8, then trace the visual encoder quantization path. Done means MXFP8 is intentionally supported there, or its unsupported status and customization path are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100