huggingface / huggingface/optimum-nvidia
How to use TensorRT model converter
Open
- Dominant language
- Python
- Stars
- 1k
- Forks
- 103
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
Referring to [src/optimum/nvidia/export/converter.py] -> class 'TensorRTModelConverter' this could 'Take a local model and create the TRTLLM checkpoint and engine'
Questions:
- What are applicable local model format? e.g. JAX, HuggingFace, DeepSpeed
- How to use this script individually to generate TRTLLM checkpoint/engine? Could you please share if any tutorial?
Thank you.
Contributor guide
Assessment
This issue has not been assessed yet.