huggingface/optimum-nvidia

View on GitHub

Stars: 1k
Forks: 103
Open beginner issues: 0
Indexed issues: 54
Avg merge: 2m
Merged PRs (30d): 1

Dominant language: Python
License: Apache-2.0
Last GitHub push: Apr 3, 2026
Latest indexed: Sep 17, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Open All

54 open issues indexed

Supporting Bert / Roberta - tags: enchancement / new_model Open

model

huggingface/optimum-nvidia#21 · 0 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
GPTQ support? Open

enhancement

huggingface/optimum-nvidia#26 · 11 comments · 2 reactions · 1 assignee · Indexed Sep 17, 2026
Excellent project, hopefully ChatGLM3 will be supported. Open

huggingface/optimum-nvidia#33 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Enhancing Compatibility and Extending Support for Optimum-NVIDIA Across Diverse Workloads Open

huggingface/optimum-nvidia#34 · 0 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
When is the pip install command coming? Open

huggingface/optimum-nvidia#36 · 2 comments · 17 reactions · 0 assignees · Indexed Sep 17, 2026
roubleshooting installation issues with Optimum-NVIDIA without docker Open

huggingface/optimum-nvidia#37 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Segmentation fault: address not mapped to object at address 0xb1fe8 Open

huggingface/optimum-nvidia#44 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
FileNotFoundError: [Errno 2] No such file or directory: '/data/Dilip/models/llama-2-7b-chat-hf/build.json' Open

bug

huggingface/optimum-nvidia#47 · 8 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
Use FP8 by default when on a supported device Open

huggingface/optimum-nvidia#50 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate` Open

bug

huggingface/optimum-nvidia#56 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
RuntimeError: TRT Engine build failed... Open

huggingface/optimum-nvidia#58 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Installation errors Open

huggingface/optimum-nvidia#59 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Feature request: streamer Open

enhancement

huggingface/optimum-nvidia#60 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Not able to run 'Generate' from QuickStart section Open

bug

huggingface/optimum-nvidia#61 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Pip Installation Open

huggingface/optimum-nvidia#62 · 4 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
Build failed with cuda runtime error. Open

huggingface/optimum-nvidia#64 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Mixtral support Open

huggingface/optimum-nvidia#67 · 4 comments · 8 reactions · 0 assignees · Indexed Sep 17, 2026
Error when Running LLAMA with tensor parallelism = 2 Open

bug

huggingface/optimum-nvidia#68 · 1 comment · 1 reaction · 1 assignee · Indexed Sep 17, 2026
Triton Inference Server Open

huggingface/optimum-nvidia#69 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
llama.py with fp8 is broken (inference produces garbage results) Open

huggingface/optimum-nvidia#71 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
How do you use the library in your scripts after pulling and running the Docker image? Open

huggingface/optimum-nvidia#72 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
How to build this environment without docker? Open

huggingface/optimum-nvidia#75 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Qwen Support Open

huggingface/optimum-nvidia#84 · 0 comments · 3 reactions · 0 assignees · Indexed Sep 17, 2026
Incorrect tensorrt_llm config class initialization Open

bug

huggingface/optimum-nvidia#90 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Original model configuration (config.json) was not found error during running inference using "Llama-2-7b-chat-hf" Open

huggingface/optimum-nvidia#91 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
can't build optimum-nvidia Open

huggingface/optimum-nvidia#96 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Add `optimum-cli export tensorrt-llm` Open

huggingface/optimum-nvidia#99 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Avoid writting engines in `.cache/huggingface/hub` Open

huggingface/optimum-nvidia#100 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Instructions on how to set TP/PP Open

huggingface/optimum-nvidia#102 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
When can I support llava Open

huggingface/optimum-nvidia#106 · 0 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
FileNotFoundError: [Errno 2] No such file or directory: 'trtllm-build' Open

huggingface/optimum-nvidia#116 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
ValueError: mutable default <class 'tensorrt_llm.lora_manager.LoraBuildConfig'> for field lora_config is not allowed: use default_factory Open

huggingface/optimum-nvidia#119 · 5 comments · 3 reactions · 0 assignees · Indexed Sep 17, 2026
Failed to import optimum.nvidia Open

huggingface/optimum-nvidia#121 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Docker container fails on RTX A6000 Open

huggingface/optimum-nvidia#122 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Is there support for StoppingCriteria? Open

enhancement

huggingface/optimum-nvidia#123 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Load from local path? Open

huggingface/optimum-nvidia#126 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Can't Run README Code Open

huggingface/optimum-nvidia#127 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
No engine file found for LLama 3 and Cuda API error with LLama 2 with use_fp8 Open

huggingface/optimum-nvidia#128 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Providing input_embeddings for generation instead of IDs Open

huggingface/optimum-nvidia#129 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Model of type gpt_bigcode is not supported Open

huggingface/optimum-nvidia#134 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Model of type falcon is not supported Open

huggingface/optimum-nvidia#140 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! Open

huggingface/optimum-nvidia#141 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Error for gated model access despite valid HF_TOKEN Open

huggingface/optimum-nvidia#142 · 0 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
Error on Quickstart example Open

huggingface/optimum-nvidia#143 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
Unable to install `optimum-nvidia` on Ubuntu Open

huggingface/optimum-nvidia#144 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
OutOfMemory - Not able to run the text-generation.py example on V100, and A10G cores. Open

huggingface/optimum-nvidia#146 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
How to use TensorRT model converter Open

huggingface/optimum-nvidia#149 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Speed up training? Open

huggingface/optimum-nvidia#161 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Is this still maintained? Open

huggingface/optimum-nvidia#168 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
How to run whisper after #133 Open

huggingface/optimum-nvidia#176 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026

huggingface/optimum-nvidia

Get new issues in your inbox